Jul 10 20:08:31.366027 ip-10-0-133-106 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 20:08:33.028694 ip-10-0-133-106 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 20:08:33.028694 ip-10-0-133-106 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 20:08:33.028694 ip-10-0-133-106 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 20:08:33.028694 ip-10-0-133-106 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 20:08:33.028694 ip-10-0-133-106 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 20:08:33.029594 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.029507    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 20:08:33.034057 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034041    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:08:33.034057 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034058    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034061    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034065    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034068    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034071    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034074    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034077    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034079    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034082    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034084    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034087    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034090    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034093    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034096    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034099    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034101    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034104    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034106    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034109    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034118    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:08:33.034121 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034120    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034125    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034129    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034132    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034135    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034138    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034140    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034143    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034145    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034148    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034150    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034153    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034156    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034159    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034161    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034164    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034166    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034169    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034171    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:08:33.034743 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034174    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034177    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034179    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034181    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034185    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034188    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034191    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034194    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034197    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034200    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034202    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034205    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034207    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034210    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034212    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034216    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034219    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034222    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034224    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:08:33.035484 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034227    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034229    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034232    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034234    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034236    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034239    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034242    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034246    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034250    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034253    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034256    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034259    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034262    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034264    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034267    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034270    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034272    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034275    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034278    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:08:33.036158 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034280    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034283    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034285    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034288    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034290    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034293    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034295    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034298    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034336    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034396    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034402    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034407    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034411    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034416    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034420    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034424    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034430    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034434    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034438    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034443    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:08:33.036623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.034448    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036433    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036444    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036448    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036451    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036454    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036458    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036461    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036463    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036466    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036470    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036472    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036475    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036477    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036480    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036482    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036485    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036487    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036490    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036492    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:08:33.037125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036495    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036497    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036500    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036503    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036505    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036507    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036510    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036513    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036515    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036518    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036521    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036523    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036526    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036533    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036537    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036540    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036543    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036545    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036548    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036551    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:08:33.037648 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036554    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036556    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036558    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036561    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036563    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036566    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036568    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036571    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036573    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036576    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036578    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036582    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036587    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036591    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036596    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036605    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036608    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036611    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036613    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:08:33.038147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036616    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036619    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036621    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036624    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036627    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036630    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036633    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036636    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036639    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036642    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036644    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036647    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036650    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036652    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036655    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036657    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036660    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036663    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036666    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036685    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:08:33.038637 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036688    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036691    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036694    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036696    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036699    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036703    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036706    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036708    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036711    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036713    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036716    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036719    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036722    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036726    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036729    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036732    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036735    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036737    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036740    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:08:33.039157 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036745    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.036748    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037176    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037181    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037185    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037190    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037194    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037197    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037200    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037203    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037205    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037208    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037210    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037213    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037215    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037218    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037221    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037223    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037226    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:08:33.039658 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037229    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037231    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037234    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037236    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037239    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037241    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037243    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037246    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037248    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037251    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037254    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037256    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037259    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037263    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037265    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037268    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037271    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037273    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037276    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037279    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:08:33.040142 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037281    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037285    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037287    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037290    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037292    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037295    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037297    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037300    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037302    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037304    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037307    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037309    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037312    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037314    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037316    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037319    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037321    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037324    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037326    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037328    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:08:33.040623 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037331    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037333    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037336    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037338    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037340    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037344    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037347    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037349    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037352    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037354    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037357    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037359    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037363    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037365    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037370    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037374    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037376    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037379    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:08:33.041147 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037383    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037386    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037389    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037392    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037394    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037397    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037399    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037401    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037404    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037407    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037409    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037411    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037414    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037416    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037419    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037421    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037424    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037426    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037429    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037432    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:08:33.041600 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037435    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037437    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037440    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.037442    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037514    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037525    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037532    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037537    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037543    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037548    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037555    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037561    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037566    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037569    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037572    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037575    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037578    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037581    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037584    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037587    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037590    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037593    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037598    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037601    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 20:08:33.042106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037604    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037607    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037610    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037614    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037617    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037621    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037624    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037627    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037630    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037633    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037636    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037639    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037643    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037646    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037649    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037653    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037656    2580 flags.go:64] FLAG: --enable-server="true"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037659    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037664    2580 flags.go:64] FLAG: --event-burst="100"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037682    2580 flags.go:64] FLAG: --event-qps="50"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037687    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037691    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037695    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037699    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 20:08:33.042768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037702    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037704    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037708    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037710    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037713    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037716    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037719    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037722    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037724    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037727    2580 flags.go:64] FLAG: --feature-gates=""
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037731    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037734    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037737    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037740    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037743    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037748    2580 flags.go:64] FLAG: --help="false"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037751    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037755    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037758    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037761    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037764    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037767    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037770    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 20:08:33.043361 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037773    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037776    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037780    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037783    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037785    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037788    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037791    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037794    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037797    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037800    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037802    2580 flags.go:64] FLAG: --lock-file=""
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037805    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037808    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037811    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037817    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037820    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037823    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037826    2580 flags.go:64] FLAG: --logging-format="text"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037828    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037832    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037835    2580 flags.go:64] FLAG: --manifest-url=""
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037837    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037842    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037845    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037851    2580 flags.go:64] FLAG: --max-pods="110"
Jul 10 20:08:33.043974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037854    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037856    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037859    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037863    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037866    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037869    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037871    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037878    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037881    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037884    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037888    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037891    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037896    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037899    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037902    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037904    2580 flags.go:64] FLAG: --port="10250"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037907    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037910    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-026830a80586f0c1e"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037913    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037916    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037919    2580 flags.go:64] FLAG: --register-node="true"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037922    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037925    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037928    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 10 20:08:33.044592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037931    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037934    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037940    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037943    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037946    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037949    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037952    2580 flags.go:64] FLAG: --runonce="false"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037955    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037960    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037963    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037966    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037969    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037972    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037975    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037978    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037981    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037984    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037987    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037989    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037993    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037996    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.037999    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038004    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038006    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038009    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 20:08:33.045185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038013    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038016    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038019    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038021    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038024    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038029    2580 flags.go:64] FLAG: --v="2"
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038033    2580 flags.go:64] FLAG: --version="false"
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038037    2580 flags.go:64] FLAG: --vmodule=""
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038041    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038044    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038155    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038159    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038162    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038165    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038168    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038173    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038175    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038178    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038180    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038183    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038185    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038188    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:08:33.045806 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038190    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038193    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038196    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038200    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038204    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038208    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038212    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038216    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038219    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038223    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038225    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038228    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038231    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038233    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038236    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038241    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038243    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038246    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038249    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:08:33.046365 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038252    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038255    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038258    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038261    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038263    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038266    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038270    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038273    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038278    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038281    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038301    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038304    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038307    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038309    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038312    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038314    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038317    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038320    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038323    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038326    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:08:33.046843 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038329    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038332    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038334    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038336    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038339    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038341    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038344    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038347    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038351    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038353    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038356    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038358    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038361    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038364    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038366    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038369    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038371    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038374    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038379    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:08:33.047415 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038381    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038385    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038389    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038392    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038395    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038397    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038400    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038402    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038405    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038408    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038410    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038413    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038416    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038419    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038422    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038424    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038427    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038429    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038432    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:08:33.047963 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038435    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038437    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038442    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038444    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038447    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038449    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038452    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038454    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038457    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.038460    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.038995    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.045243    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.045260    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.045971    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 20:08:33.048418 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.045979    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046038    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046043    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046046    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046049    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046053    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046056    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046059    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046062    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046064    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046067    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046070    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046072    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046075    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046078    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046081    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046083    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046086    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046088    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046091    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:08:33.048791 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046094    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046096    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046099    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046101    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046104    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046106    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046109    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046112    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046114    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046117    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046122    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046125    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046127    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046132    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046135    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046138    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046141    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046144    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046147    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:08:33.049283 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046149    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046152    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046156    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046160    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046163    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046166    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046169    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046173    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046176    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046178    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046181    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046183    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046186    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046189    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046191    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046194    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046196    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046199    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046201    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:08:33.049741 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046204    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046207    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046209    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046212    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046215    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046218    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046220    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046223    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046226    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046228    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046230    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046233    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046235    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046238    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046240    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046243    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046245    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046248    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046250    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046252    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:08:33.050196 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046255    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046257    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046260    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046263    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046265    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046268    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046270    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046273    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046275    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046277    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046280    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046282    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046285    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046287    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046290    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046292    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046296    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046298    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046302    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046304    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:08:33.050758 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046307    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046309    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.046314    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046423    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046428    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046430    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046433    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046436    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046439    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046442    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046445    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046447    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046450    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046452    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046455    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 20:08:33.051264 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046457    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046460    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046462    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046465    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046468    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046470    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046472    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046475    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046477    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046480    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046482    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046485    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046487    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046491    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046494    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046497    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046500    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046502    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046505    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046507    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 20:08:33.051635 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046510    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046512    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046515    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046517    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046520    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046522    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046525    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046527    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046530    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046534    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046538    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046540    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046543    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046546    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046549    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046552    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046554    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046557    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046560    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 20:08:33.052125 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046562    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046565    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046568    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046570    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046572    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046575    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046578    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046580    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046583    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046585    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046588    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046591    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046593    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046597    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046600    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046602    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046604    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046607    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046609    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 20:08:33.052590 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046612    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046614    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046617    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046619    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046622    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046624    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046626    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046629    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046632    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046634    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046638    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046641    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046643    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046646    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046649    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046651    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046654    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046656    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046659    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046661    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 20:08:33.053065 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046679    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046682    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046685    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046687    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046691    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046693    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046696    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046698    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:33.046701    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.046706    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.046815    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.048627    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.049728    2580 server.go:1019] "Starting client certificate rotation"
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.049825    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 20:08:33.053564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.050448    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 20:08:33.053944 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.053408    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 20:08:33.054994 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.054978    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 20:08:33.064202 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.064185    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 10 20:08:33.066065 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.066050    2580 log.go:25] "Validated CRI v1 image API"
Jul 10 20:08:33.066907 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.066893    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 20:08:33.069704 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.069685    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b5f4d284-5147-446b-85f1-51531200389e:/dev/nvme0n1p3 fac167d3-a71e-45a0-8a0f-015a2b4e75a1:/dev/nvme0n1p4]
Jul 10 20:08:33.069773 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.069703    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 20:08:33.075283 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.075055    2580 manager.go:216] Machine: {Timestamp:2026-07-10 20:08:33.073315387 +0000 UTC m=+0.299082306 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101460 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec226680d616af3bb80be459151a6669 SystemUUID:ec226680-d616-af3b-b80b-e459151a6669 BootID:1ee5326b-63c0-4599-bec3-5f4f07d953e1 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ac:b7:59:7d:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ac:b7:59:7d:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:f1:11:74:81:8b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 20:08:33.075283 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.075282    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 20:08:33.075388 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.075361    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 20:08:33.078120 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.078090    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 20:08:33.078264 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.078122    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-106.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 20:08:33.079143 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.079132    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 20:08:33.079185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.079144    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 20:08:33.079185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.079159    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 20:08:33.079423 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.079406    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 20:08:33.079696 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.079683    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 20:08:33.080728 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.080718    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 20:08:33.084038 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084025    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 20:08:33.084171 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084160    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 20:08:33.084204 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084189    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 20:08:33.084242 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084206    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 20:08:33.084242 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084223    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 20:08:33.084242 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084233    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 10 20:08:33.084242 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084243    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 20:08:33.084603 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084592    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 20:08:33.084638 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.084611    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 20:08:33.086607 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.086589    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 20:08:33.088150 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088137    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 20:08:33.088209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088167    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 20:08:33.088317 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088307    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 20:08:33.088353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088322    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 20:08:33.088353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088328    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 20:08:33.088353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088334    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 20:08:33.088353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088342    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 20:08:33.088353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088347    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 20:08:33.088789 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088780    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 20:08:33.088789 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088789    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 20:08:33.088858 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088796    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 20:08:33.088858 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088808    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 20:08:33.088858 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088820    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 20:08:33.088858 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.088828    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 20:08:33.090199 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.090187    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 20:08:33.090234 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.090200    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 20:08:33.093431 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.093413    2580 server.go:1268] "Started kubelet"
Jul 10 20:08:33.094021 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.093635    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 20:08:33.094021 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.093808    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 20:08:33.094442 ip-10-0-133-106 systemd[1]: Started Kubernetes Kubelet.
Jul 10 20:08:33.096111 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.095475    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 20:08:33.096551 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.096511    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 20:08:33.097624 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.097603    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 20:08:33.098238 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.098218    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-106.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 20:08:33.098324 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.098213    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 20:08:33.098403 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.098350    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 20:08:33.099956 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.099933    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 20:08:33.104462 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.104441    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 20:08:33.104462 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.104458    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 20:08:33.104603 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.104542    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.104929 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.103428    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-106.ec2.internal.18c106890bc85299  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-106.ec2.internal,UID:ip-10-0-133-106.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-106.ec2.internal,},FirstTimestamp:2026-07-10 20:08:33.093391001 +0000 UTC m=+0.319157921,LastTimestamp:2026-07-10 20:08:33.093391001 +0000 UTC m=+0.319157921,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-106.ec2.internal,}"
Jul 10 20:08:33.105096 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.105083    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 20:08:33.105217 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.105207    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 20:08:33.105297 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.105277    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 20:08:33.105357 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.105344    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 20:08:33.105411 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.105358    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 20:08:33.106791 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.105812    2580 factory.go:55] Registering systemd factory
Jul 10 20:08:33.106791 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.106275    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 10 20:08:33.106923 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.106875    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qcl86"
Jul 10 20:08:33.107122 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.107088    2580 factory.go:153] Registering CRI-O factory
Jul 10 20:08:33.107122 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.107105    2580 factory.go:223] Registration of the crio container factory successfully
Jul 10 20:08:33.107255 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.107155    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 20:08:33.107255 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.107205    2580 factory.go:103] Registering Raw factory
Jul 10 20:08:33.107255 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.107219    2580 manager.go:1196] Started watching for new ooms in manager
Jul 10 20:08:33.107682 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.107651    2580 manager.go:349] Starting recovery of all containers
Jul 10 20:08:33.114875 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.114850    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-106.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 20:08:33.115105 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.114943    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 20:08:33.116945 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.116924    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qcl86"
Jul 10 20:08:33.118525 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.118508    2580 manager.go:354] Recovery completed
Jul 10 20:08:33.122660 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.122648    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:08:33.125683 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.125648    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:08:33.125759 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.125703    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:08:33.125759 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.125717    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:08:33.126265 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.126252    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 20:08:33.126265 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.126265    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 20:08:33.126366 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.126289    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:08:33.126366 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.126317    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 20:08:33.127036 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.127021    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:08:33.127114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.127054    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:08:33.127114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.127069    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:08:33.127606 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.127544    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-106.ec2.internal.18c106890db5137e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-106.ec2.internal,UID:ip-10-0-133-106.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-106.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-106.ec2.internal,},FirstTimestamp:2026-07-10 20:08:33.125684094 +0000 UTC m=+0.351451016,LastTimestamp:2026-07-10 20:08:33.125684094 +0000 UTC m=+0.351451016,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-106.ec2.internal,}"
Jul 10 20:08:33.128299 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.128286    2580 policy_none.go:49] "None policy: Start"
Jul 10 20:08:33.128339 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.128306    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 20:08:33.128339 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.128321    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 20:08:33.130177 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.130167    2580 policy_none.go:47] "Start"
Jul 10 20:08:33.174247 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.174230    2580 manager.go:337] "Starting Device Plugin manager"
Jul 10 20:08:33.174339 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.174275    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 20:08:33.174339 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.174288    2580 server.go:87] "Starting device plugin registration server"
Jul 10 20:08:33.174568 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.174551    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 20:08:33.174657 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.174564    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 20:08:33.174727 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.174664    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 20:08:33.174776 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.174764    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 20:08:33.174776 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.174775    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 20:08:33.175518 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.175499    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 20:08:33.175606 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.175552    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.273261 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.273220    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 20:08:33.275370 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.274462    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 20:08:33.275370 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.274497    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 20:08:33.275370 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.274523    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 20:08:33.275370 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.274568    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 20:08:33.275370 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.274662    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:08:33.275789 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.275773    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:08:33.275881 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.275808    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:08:33.275881 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.275825    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:08:33.275881 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.275854    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.277053 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.277037    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:08:33.288058 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.288001    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.288058 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.288028    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-106.ec2.internal\": node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.338140 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.338111    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.374912 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.374872    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"]
Jul 10 20:08:33.375003 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.374959    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:08:33.375878 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.375860    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:08:33.375949 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.375895    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:08:33.375949 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.375909    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:08:33.378212 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378199    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:08:33.378381 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.378419 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378397    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:08:33.378939 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378922    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:08:33.379037 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378922    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:08:33.379037 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378980    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:08:33.379037 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378993    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:08:33.379037 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.378952    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:08:33.379037 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.379029    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:08:33.381159 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.381147    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.381211 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.381169    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 20:08:33.381844 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.381827    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 20:08:33.381895 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.381860    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 20:08:33.381895 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.381873    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeHasSufficientPID"
Jul 10 20:08:33.404287 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.404269    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-106.ec2.internal\" not found" node="ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.406828 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.406809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d5f9721c0a1f950164b033209b6a13a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"d5f9721c0a1f950164b033209b6a13a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.406897 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.406842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5f9721c0a1f950164b033209b6a13a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"d5f9721c0a1f950164b033209b6a13a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.406897 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.406868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4dc4522d52b12e6a46164476b50912d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-106.ec2.internal\" (UID: \"b4dc4522d52b12e6a46164476b50912d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.408769 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.408756    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-106.ec2.internal\" not found" node="ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.438225 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.438205    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.507123 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.507096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d5f9721c0a1f950164b033209b6a13a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"d5f9721c0a1f950164b033209b6a13a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.507212 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.507128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5f9721c0a1f950164b033209b6a13a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"d5f9721c0a1f950164b033209b6a13a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.507212 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.507147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4dc4522d52b12e6a46164476b50912d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-106.ec2.internal\" (UID: \"b4dc4522d52b12e6a46164476b50912d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.507212 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.507191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5f9721c0a1f950164b033209b6a13a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"d5f9721c0a1f950164b033209b6a13a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.507212 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.507202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d5f9721c0a1f950164b033209b6a13a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal\" (UID: \"d5f9721c0a1f950164b033209b6a13a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.507348 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.507230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4dc4522d52b12e6a46164476b50912d-config\") pod \"kube-apiserver-proxy-ip-10-0-133-106.ec2.internal\" (UID: \"b4dc4522d52b12e6a46164476b50912d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.539303 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.539234    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.640034 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.640010    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.706321 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.706285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.711065 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.710965    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:33.740444 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.740406    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.841004 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:33.840902    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-106.ec2.internal\" not found"
Jul 10 20:08:33.924195 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:33.924167    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:08:34.004948 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.004912    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:34.017058 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.017038    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 20:08:34.018703 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.018687    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal"
Jul 10 20:08:34.026097 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.026079    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 20:08:34.049138 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.049119    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 20:08:34.049774 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.049241    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 20:08:34.049774 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.049274    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 20:08:34.085417 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.085392    2580 apiserver.go:52] "Watching apiserver"
Jul 10 20:08:34.103994 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.103596    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 20:08:34.104760 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.104733    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 20:08:34.105287 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.105263    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-cbbbn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal","openshift-multus/network-metrics-daemon-4wx2c","openshift-ovn-kubernetes/ovnkube-node-dqm9m","kube-system/konnectivity-agent-g7kn9","kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4","openshift-image-registry/node-ca-zzsd6","openshift-multus/multus-additional-cni-plugins-nzdm9","openshift-multus/multus-hpfkh","openshift-network-diagnostics/network-check-target-lzt8q","openshift-network-operator/iptables-alerter-bclrh","kube-system/global-pull-secret-syncer-szwhd","openshift-cluster-node-tuning-operator/tuned-l9rxz"]
Jul 10 20:08:34.107761 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.107744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.109758 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.109740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.109863 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.109799    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:34.110019 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 20:08:34.110076 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110040    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 20:08:34.110132 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110042    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-946ng\""
Jul 10 20:08:34.110132 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110083    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 20:08:34.110209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.110209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-registration-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.110298 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-device-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.110298 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.110298 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-socket-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.110416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-sys-fs\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.110416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.110338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vft28\" (UniqueName: \"kubernetes.io/projected/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-kube-api-access-vft28\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.111730 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.111719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:34.111775 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.111763    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:34.114004 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.113989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.116129 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116111    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.116333 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116317    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 20:08:34.116519 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116492    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 20:08:34.116621 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116554    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 20:08:34.116621 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116558    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 20:08:34.116621 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116615    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 20:08:34.116763 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116655    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 20:08:34.116763 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116756    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6hl5f\""
Jul 10 20:08:34.116832 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.116777    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 20:08:34.118373 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.118355    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 20:08:34.118373 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.118370    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vtwfw\""
Jul 10 20:08:34.118512 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.118391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 20:08:34.118512 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.118455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.120556 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.120520    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 20:03:33 +0000 UTC" deadline="2028-01-25 12:27:11.430862776 +0000 UTC"
Jul 10 20:08:34.120556 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.120553    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13528h18m37.310313s"
Jul 10 20:08:34.120721 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.120708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cjp5w\""
Jul 10 20:08:34.120790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.120769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.121026 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.121007    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 20:08:34.121107 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.121085    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:08:34.123029 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.123010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.123150 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.123136    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 20:08:34.123359 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.123328    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 20:08:34.123487 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.123471    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r4dgd\""
Jul 10 20:08:34.123565 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.123550    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 20:08:34.123656 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.123640    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 20:08:34.123724 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.123640    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 20:08:34.125289 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.125272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.125384 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.125368    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 20:08:34.125622 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.125606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 20:08:34.125871 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.125852    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t9458\""
Jul 10 20:08:34.127610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.127580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.127610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.127594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 20:08:34.127610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.127602    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qlq7z\""
Jul 10 20:08:34.127866 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.127594    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 20:08:34.127866 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.127843    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 20:08:34.130630 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.130178    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 20:08:34.130630 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.130196    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-29ffz\""
Jul 10 20:08:34.130833 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.130754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:34.130915 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.130876    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:34.133610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.133595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.135880 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.135859    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 20:08:34.135970 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.135931    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 20:08:34.135970 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.135946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:08:34.136053 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.135974    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7mn2p\""
Jul 10 20:08:34.146648 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.146630    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2cgkn"
Jul 10 20:08:34.152603 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.152591    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2cgkn"
Jul 10 20:08:34.206906 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.206882    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 20:08:34.211199 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.211326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.211326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-cni-netd\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.211326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-kubernetes\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.211326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-etc-selinux\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.211326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-cnibin\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-k8s-cni-cncf-io\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2dacc1cf-8421-46ab-9e02-8a3db6053886-iptables-alerter-script\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-system-cni-dir\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-sys\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-system-cni-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-cni-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/222d6857-2959-42ad-b364-7322ba3bcb8e-cni-binary-copy\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.211571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2dacc1cf-8421-46ab-9e02-8a3db6053886-host-slash\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-cni-bin\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovnkube-script-lib\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-lib-modules\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8rwm\" (UniqueName: \"kubernetes.io/projected/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-kube-api-access-g8rwm\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-ovn\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-host\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5gcd\" (UniqueName: \"kubernetes.io/projected/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-kube-api-access-b5gcd\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-dbus\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211837    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44056a08-41c4-4faf-a8d3-d66637bef925-konnectivity-ca\") pod \"konnectivity-agent-g7kn9\" (UID: \"44056a08-41c4-4faf-a8d3-d66637bef925\") " pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66dnx\" (UniqueName: \"kubernetes.io/projected/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-kube-api-access-66dnx\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d89da9d9-5f26-4757-90ee-cf3daed84967-host\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211889    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-cni-multus\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntjwr\" (UniqueName: \"kubernetes.io/projected/222d6857-2959-42ad-b364-7322ba3bcb8e-kube-api-access-ntjwr\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.211957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-var-lib-kubelet\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211953    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vft28\" (UniqueName: \"kubernetes.io/projected/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-kube-api-access-vft28\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211978    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.211997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-systemd\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d06df879-3247-42e7-a523-e15253ff8ede-etc-tuned\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-hosts-file\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-etc-kubernetes\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovnkube-config\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cnibin\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-run\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-socket-dir-parent\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxqff\" (UniqueName: \"kubernetes.io/projected/2dacc1cf-8421-46ab-9e02-8a3db6053886-kube-api-access-rxqff\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-os-release\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-cni-bin\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-conf-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.212915 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-socket-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-sys-fs\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-var-lib-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-run-netns\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovn-node-metrics-cert\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44056a08-41c4-4faf-a8d3-d66637bef925-agent-certs\") pod \"konnectivity-agent-g7kn9\" (UID: \"44056a08-41c4-4faf-a8d3-d66637bef925\") " pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysctl-d\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d06df879-3247-42e7-a523-e15253ff8ede-tmp\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212538    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-socket-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-hostroot\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-sys-fs\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212404    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:34.213652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-log-socket\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-env-overrides\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-os-release\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-multus-certs\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-registration-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-registration-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-kubelet-config\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-systemd-units\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-slash\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-node-log\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysconfig\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.212995    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4pgs\" (UniqueName: \"kubernetes.io/projected/d06df879-3247-42e7-a523-e15253ff8ede-kube-api-access-f4pgs\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-tmp-dir\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-kubelet\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdq49\" (UniqueName: \"kubernetes.io/projected/6f224662-2d68-4d3a-bb00-16fa779f6af4-kube-api-access-mdq49\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-modprobe-d\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d89da9d9-5f26-4757-90ee-cf3daed84967-serviceca\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.214209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-netns\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-kubelet\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-device-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-systemd\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213279    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-device-dir\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysctl-conf\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86k6l\" (UniqueName: \"kubernetes.io/projected/d89da9d9-5f26-4757-90ee-cf3daed84967-kube-api-access-86k6l\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-daemon-config\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.214700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.213411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-etc-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.217888 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.217863    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 20:08:34.220261 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.220244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vft28\" (UniqueName: \"kubernetes.io/projected/f6365594-a556-40ad-bfbf-dcdb91f5a5d8-kube-api-access-vft28\") pod \"aws-ebs-csi-driver-node-zrzr4\" (UID: \"f6365594-a556-40ad-bfbf-dcdb91f5a5d8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.303321 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.303290    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:08:34.314136 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-etc-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-cni-netd\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314172    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-kubernetes\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.314248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-cnibin\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314233    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-etc-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-cni-netd\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-k8s-cni-cncf-io\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-cnibin\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2dacc1cf-8421-46ab-9e02-8a3db6053886-iptables-alerter-script\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-kubernetes\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-k8s-cni-cncf-io\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-system-cni-dir\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-sys\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-system-cni-dir\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-sys\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-system-cni-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-cni-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/222d6857-2959-42ad-b364-7322ba3bcb8e-cni-binary-copy\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314541    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-cni-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314506    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-system-cni-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2dacc1cf-8421-46ab-9e02-8a3db6053886-host-slash\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-cni-bin\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovnkube-script-lib\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2dacc1cf-8421-46ab-9e02-8a3db6053886-host-slash\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-lib-modules\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314694    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-cni-bin\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8rwm\" (UniqueName: \"kubernetes.io/projected/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-kube-api-access-g8rwm\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:34.314756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-ovn\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-host\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5gcd\" (UniqueName: \"kubernetes.io/projected/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-kube-api-access-b5gcd\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-dbus\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314829    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-lib-modules\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44056a08-41c4-4faf-a8d3-d66637bef925-konnectivity-ca\") pod \"konnectivity-agent-g7kn9\" (UID: \"44056a08-41c4-4faf-a8d3-d66637bef925\") " pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314906    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66dnx\" (UniqueName: \"kubernetes.io/projected/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-kube-api-access-66dnx\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-host\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d89da9d9-5f26-4757-90ee-cf3daed84967-host\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-ovn\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-cni-multus\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.314996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntjwr\" (UniqueName: \"kubernetes.io/projected/222d6857-2959-42ad-b364-7322ba3bcb8e-kube-api-access-ntjwr\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315008    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-run-ovn-kubernetes\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-var-lib-kubelet\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315042    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:34.315478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-dbus\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-cni-multus\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d89da9d9-5f26-4757-90ee-cf3daed84967-host\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/222d6857-2959-42ad-b364-7322ba3bcb8e-cni-binary-copy\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-var-lib-kubelet\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-systemd\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d06df879-3247-42e7-a523-e15253ff8ede-etc-tuned\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-hosts-file\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-systemd\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.315270    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-hosts-file\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-etc-kubernetes\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-etc-kubernetes\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.315347    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs podName:d9f1c3d3-65c9-4365-8dc8-0517ceae9708 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:34.815327831 +0000 UTC m=+2.041094766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs") pod "network-metrics-daemon-4wx2c" (UID: "d9f1c3d3-65c9-4365-8dc8-0517ceae9708") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovnkube-config\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cnibin\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-run\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.316286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-socket-dir-parent\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cnibin\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxqff\" (UniqueName: \"kubernetes.io/projected/2dacc1cf-8421-46ab-9e02-8a3db6053886-kube-api-access-rxqff\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/44056a08-41c4-4faf-a8d3-d66637bef925-konnectivity-ca\") pod \"konnectivity-agent-g7kn9\" (UID: \"44056a08-41c4-4faf-a8d3-d66637bef925\") " pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-os-release\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315548    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-run\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315565    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-cni-bin\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-cni-bin\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-conf-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-conf-dir\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-socket-dir-parent\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-os-release\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2dacc1cf-8421-46ab-9e02-8a3db6053886-iptables-alerter-script\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-var-lib-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-run-netns\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.317194 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovn-node-metrics-cert\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44056a08-41c4-4faf-a8d3-d66637bef925-agent-certs\") pod \"konnectivity-agent-g7kn9\" (UID: \"44056a08-41c4-4faf-a8d3-d66637bef925\") " pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-var-lib-openvswitch\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.315771    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.315920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovnkube-config\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.315987    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret podName:eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:34.81595419 +0000 UTC m=+2.041721111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret") pod "global-pull-secret-syncer-szwhd" (UID: "eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316010    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysctl-d\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d06df879-3247-42e7-a523-e15253ff8ede-tmp\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-hostroot\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovnkube-script-lib\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-log-socket\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316182    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-env-overrides\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-hostroot\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-os-release\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.318108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysctl-d\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-multus-certs\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-kubelet-config\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-systemd-units\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-os-release\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-slash\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-node-log\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysconfig\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-kubelet-config\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316492    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4pgs\" (UniqueName: \"kubernetes.io/projected/d06df879-3247-42e7-a523-e15253ff8ede-kube-api-access-f4pgs\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-multus-certs\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-tmp-dir\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-systemd-units\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316538    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-kubelet\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdq49\" (UniqueName: \"kubernetes.io/projected/6f224662-2d68-4d3a-bb00-16fa779f6af4-kube-api-access-mdq49\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysconfig\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-modprobe-d\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d89da9d9-5f26-4757-90ee-cf3daed84967-serviceca\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.318737 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316611    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6f224662-2d68-4d3a-bb00-16fa779f6af4-env-overrides\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-netns\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-kubelet\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-systemd\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316818    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysctl-conf\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-modprobe-d\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-var-lib-kubelet\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-node-log\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d06df879-3247-42e7-a523-e15253ff8ede-etc-sysctl-conf\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316960    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/222d6857-2959-42ad-b364-7322ba3bcb8e-host-run-netns\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-kubelet\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-run-systemd\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-log-socket\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d89da9d9-5f26-4757-90ee-cf3daed84967-serviceca\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86k6l\" (UniqueName: \"kubernetes.io/projected/d89da9d9-5f26-4757-90ee-cf3daed84967-kube-api-access-86k6l\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-daemon-config\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-tmp-dir\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.319420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.316440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-slash\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317285    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6f224662-2d68-4d3a-bb00-16fa779f6af4-host-run-netns\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.319947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.319947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/222d6857-2959-42ad-b364-7322ba3bcb8e-multus-daemon-config\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.319947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.317954    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d06df879-3247-42e7-a523-e15253ff8ede-etc-tuned\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.319947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.318633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/44056a08-41c4-4faf-a8d3-d66637bef925-agent-certs\") pod \"konnectivity-agent-g7kn9\" (UID: \"44056a08-41c4-4faf-a8d3-d66637bef925\") " pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.319947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.318925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d06df879-3247-42e7-a523-e15253ff8ede-tmp\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.319947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.319619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f224662-2d68-4d3a-bb00-16fa779f6af4-ovn-node-metrics-cert\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.320898 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.320879    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:08:34.325622 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.325596    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:08:34.325622 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.325615    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:08:34.325622 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.325627    2580 projected.go:196] Error preparing data for projected volume kube-api-access-kwtbx for pod openshift-network-diagnostics/network-check-target-lzt8q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:34.325939 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.325686    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx podName:67a7d692-17e3-4349-a6eb-78a4361a27ce nodeName:}" failed. No retries permitted until 2026-07-10 20:08:34.82565779 +0000 UTC m=+2.051424717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kwtbx" (UniqueName: "kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx") pod "network-check-target-lzt8q" (UID: "67a7d692-17e3-4349-a6eb-78a4361a27ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:34.327467 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.327440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntjwr\" (UniqueName: \"kubernetes.io/projected/222d6857-2959-42ad-b364-7322ba3bcb8e-kube-api-access-ntjwr\") pod \"multus-hpfkh\" (UID: \"222d6857-2959-42ad-b364-7322ba3bcb8e\") " pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.327550 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.327509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5gcd\" (UniqueName: \"kubernetes.io/projected/edfc10a5-c760-4fb3-8ed1-49e0bffd7f80-kube-api-access-b5gcd\") pod \"node-resolver-cbbbn\" (UID: \"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80\") " pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.327851 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.327835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66dnx\" (UniqueName: \"kubernetes.io/projected/8620a41e-4cb4-4a0b-b54d-ef177115f6b3-kube-api-access-66dnx\") pod \"multus-additional-cni-plugins-nzdm9\" (UID: \"8620a41e-4cb4-4a0b-b54d-ef177115f6b3\") " pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.328112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.328095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxqff\" (UniqueName: \"kubernetes.io/projected/2dacc1cf-8421-46ab-9e02-8a3db6053886-kube-api-access-rxqff\") pod \"iptables-alerter-bclrh\" (UID: \"2dacc1cf-8421-46ab-9e02-8a3db6053886\") " pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.328257 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.328237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8rwm\" (UniqueName: \"kubernetes.io/projected/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-kube-api-access-g8rwm\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:34.332047 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.332021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdq49\" (UniqueName: \"kubernetes.io/projected/6f224662-2d68-4d3a-bb00-16fa779f6af4-kube-api-access-mdq49\") pod \"ovnkube-node-dqm9m\" (UID: \"6f224662-2d68-4d3a-bb00-16fa779f6af4\") " pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.332162 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.332063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4pgs\" (UniqueName: \"kubernetes.io/projected/d06df879-3247-42e7-a523-e15253ff8ede-kube-api-access-f4pgs\") pod \"tuned-l9rxz\" (UID: \"d06df879-3247-42e7-a523-e15253ff8ede\") " pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.332286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.332254    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86k6l\" (UniqueName: \"kubernetes.io/projected/d89da9d9-5f26-4757-90ee-cf3daed84967-kube-api-access-86k6l\") pod \"node-ca-zzsd6\" (UID: \"d89da9d9-5f26-4757-90ee-cf3daed84967\") " pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.344591 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.344432    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4dc4522d52b12e6a46164476b50912d.slice/crio-2c8ad3a81375709cfb1a32fa198d9958aa8738f1315a98bcbd73cfb12227540f WatchSource:0}: Error finding container 2c8ad3a81375709cfb1a32fa198d9958aa8738f1315a98bcbd73cfb12227540f: Status 404 returned error can't find the container with id 2c8ad3a81375709cfb1a32fa198d9958aa8738f1315a98bcbd73cfb12227540f
Jul 10 20:08:34.348292 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.348278    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:08:34.428429 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.428350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4"
Jul 10 20:08:34.437995 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.437964    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6365594_a556_40ad_bfbf_dcdb91f5a5d8.slice/crio-66c2a39b37157e2d7a80d151dcc257390188e47f04dee86bc0cc072b9df87b61 WatchSource:0}: Error finding container 66c2a39b37157e2d7a80d151dcc257390188e47f04dee86bc0cc072b9df87b61: Status 404 returned error can't find the container with id 66c2a39b37157e2d7a80d151dcc257390188e47f04dee86bc0cc072b9df87b61
Jul 10 20:08:34.442339 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.442319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:08:34.447349 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.447331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:08:34.451304 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.451284    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f224662_2d68_4d3a_bb00_16fa779f6af4.slice/crio-eeb3e6093e95b9fd094cba24ad9a0653a10fdfc753bfa78fcde26ca085ac92ea WatchSource:0}: Error finding container eeb3e6093e95b9fd094cba24ad9a0653a10fdfc753bfa78fcde26ca085ac92ea: Status 404 returned error can't find the container with id eeb3e6093e95b9fd094cba24ad9a0653a10fdfc753bfa78fcde26ca085ac92ea
Jul 10 20:08:34.456307 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.456283    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44056a08_41c4_4faf_a8d3_d66637bef925.slice/crio-b3310ca41436e3e4f913350190875983173ce54e3fd6c77077b9d8ff91a3fa0f WatchSource:0}: Error finding container b3310ca41436e3e4f913350190875983173ce54e3fd6c77077b9d8ff91a3fa0f: Status 404 returned error can't find the container with id b3310ca41436e3e4f913350190875983173ce54e3fd6c77077b9d8ff91a3fa0f
Jul 10 20:08:34.481042 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.481020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l9rxz"
Jul 10 20:08:34.491578 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.491554    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd06df879_3247_42e7_a523_e15253ff8ede.slice/crio-ca3df6498bb55931a928cc4411dd33ebde4932f135670e032d5b80b58e6f5fe4 WatchSource:0}: Error finding container ca3df6498bb55931a928cc4411dd33ebde4932f135670e032d5b80b58e6f5fe4: Status 404 returned error can't find the container with id ca3df6498bb55931a928cc4411dd33ebde4932f135670e032d5b80b58e6f5fe4
Jul 10 20:08:34.494426 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.494377    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nzdm9"
Jul 10 20:08:34.501552 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.501531    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8620a41e_4cb4_4a0b_b54d_ef177115f6b3.slice/crio-742eee3d7034b05dc6cdba8a89c9cb0035cdffe7db5c1a86f4ee8e558ba7dba2 WatchSource:0}: Error finding container 742eee3d7034b05dc6cdba8a89c9cb0035cdffe7db5c1a86f4ee8e558ba7dba2: Status 404 returned error can't find the container with id 742eee3d7034b05dc6cdba8a89c9cb0035cdffe7db5c1a86f4ee8e558ba7dba2
Jul 10 20:08:34.509312 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.509282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cbbbn"
Jul 10 20:08:34.521988 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.521962    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedfc10a5_c760_4fb3_8ed1_49e0bffd7f80.slice/crio-d6c2e817ff1d42388a8f2a1134811da07d1c20b499db7519a50112d2e5bdd3cc WatchSource:0}: Error finding container d6c2e817ff1d42388a8f2a1134811da07d1c20b499db7519a50112d2e5bdd3cc: Status 404 returned error can't find the container with id d6c2e817ff1d42388a8f2a1134811da07d1c20b499db7519a50112d2e5bdd3cc
Jul 10 20:08:34.523115 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.523096    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f9721c0a1f950164b033209b6a13a6.slice/crio-8d00e1cc54d8ac768f50b8f36e9597ff1d23b1fad2e8a0fe59b579ebd8ef753d WatchSource:0}: Error finding container 8d00e1cc54d8ac768f50b8f36e9597ff1d23b1fad2e8a0fe59b579ebd8ef753d: Status 404 returned error can't find the container with id 8d00e1cc54d8ac768f50b8f36e9597ff1d23b1fad2e8a0fe59b579ebd8ef753d
Jul 10 20:08:34.526054 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.526040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zzsd6"
Jul 10 20:08:34.534768 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.534751    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89da9d9_5f26_4757_90ee_cf3daed84967.slice/crio-6934b9e38774d2cab03efa082bea3c7690f7476e0341b2ef27fe62a0afb3da3f WatchSource:0}: Error finding container 6934b9e38774d2cab03efa082bea3c7690f7476e0341b2ef27fe62a0afb3da3f: Status 404 returned error can't find the container with id 6934b9e38774d2cab03efa082bea3c7690f7476e0341b2ef27fe62a0afb3da3f
Jul 10 20:08:34.553478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.553459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hpfkh"
Jul 10 20:08:34.559322 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.559306    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bclrh"
Jul 10 20:08:34.561516 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.561493    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222d6857_2959_42ad_b364_7322ba3bcb8e.slice/crio-de2cdd9cdfaca1fafcaccb42c6baf464f14012f3f8bcc8e74e7149ab70e0c536 WatchSource:0}: Error finding container de2cdd9cdfaca1fafcaccb42c6baf464f14012f3f8bcc8e74e7149ab70e0c536: Status 404 returned error can't find the container with id de2cdd9cdfaca1fafcaccb42c6baf464f14012f3f8bcc8e74e7149ab70e0c536
Jul 10 20:08:34.567494 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:08:34.567476    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dacc1cf_8421_46ab_9e02_8a3db6053886.slice/crio-9940f45596be1c5d9edf97eec9a168fc93f207467b14c48597b6775e5a3dc7e3 WatchSource:0}: Error finding container 9940f45596be1c5d9edf97eec9a168fc93f207467b14c48597b6775e5a3dc7e3: Status 404 returned error can't find the container with id 9940f45596be1c5d9edf97eec9a168fc93f207467b14c48597b6775e5a3dc7e3
Jul 10 20:08:34.821524 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.821419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:34.821524 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.821453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:34.821714 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.821587    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:34.821714 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.821647    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs podName:d9f1c3d3-65c9-4365-8dc8-0517ceae9708 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:35.821632876 +0000 UTC m=+3.047399788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs") pod "network-metrics-daemon-4wx2c" (UID: "d9f1c3d3-65c9-4365-8dc8-0517ceae9708") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:34.821714 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.821591    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:34.821821 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.821723    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret podName:eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:35.821713135 +0000 UTC m=+3.047480042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret") pod "global-pull-secret-syncer-szwhd" (UID: "eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:34.921859 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:34.921822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:34.922217 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.922045    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:08:34.922217 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.922068    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:08:34.922217 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.922082    2580 projected.go:196] Error preparing data for projected volume kube-api-access-kwtbx for pod openshift-network-diagnostics/network-check-target-lzt8q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:34.922217 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:34.922138    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx podName:67a7d692-17e3-4349-a6eb-78a4361a27ce nodeName:}" failed. No retries permitted until 2026-07-10 20:08:35.922120159 +0000 UTC m=+3.147887069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwtbx" (UniqueName: "kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx") pod "network-check-target-lzt8q" (UID: "67a7d692-17e3-4349-a6eb-78a4361a27ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:35.155644 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.155515    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:03:34 +0000 UTC" deadline="2028-04-09 23:29:26.361920714 +0000 UTC"
Jul 10 20:08:35.155644 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.155591    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15339h20m51.206333694s"
Jul 10 20:08:35.278622 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.278119    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:35.278622 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.278243    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:35.311268 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.311162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zzsd6" event={"ID":"d89da9d9-5f26-4757-90ee-cf3daed84967","Type":"ContainerStarted","Data":"6934b9e38774d2cab03efa082bea3c7690f7476e0341b2ef27fe62a0afb3da3f"}
Jul 10 20:08:35.340165 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.340127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerStarted","Data":"742eee3d7034b05dc6cdba8a89c9cb0035cdffe7db5c1a86f4ee8e558ba7dba2"}
Jul 10 20:08:35.346700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.346644    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g7kn9" event={"ID":"44056a08-41c4-4faf-a8d3-d66637bef925","Type":"ContainerStarted","Data":"b3310ca41436e3e4f913350190875983173ce54e3fd6c77077b9d8ff91a3fa0f"}
Jul 10 20:08:35.367368 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.367327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"eeb3e6093e95b9fd094cba24ad9a0653a10fdfc753bfa78fcde26ca085ac92ea"}
Jul 10 20:08:35.386000 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.385837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bclrh" event={"ID":"2dacc1cf-8421-46ab-9e02-8a3db6053886","Type":"ContainerStarted","Data":"9940f45596be1c5d9edf97eec9a168fc93f207467b14c48597b6775e5a3dc7e3"}
Jul 10 20:08:35.404216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.404139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" event={"ID":"d5f9721c0a1f950164b033209b6a13a6","Type":"ContainerStarted","Data":"8d00e1cc54d8ac768f50b8f36e9597ff1d23b1fad2e8a0fe59b579ebd8ef753d"}
Jul 10 20:08:35.406971 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.406903    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cbbbn" event={"ID":"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80","Type":"ContainerStarted","Data":"d6c2e817ff1d42388a8f2a1134811da07d1c20b499db7519a50112d2e5bdd3cc"}
Jul 10 20:08:35.438774 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.438650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l9rxz" event={"ID":"d06df879-3247-42e7-a523-e15253ff8ede","Type":"ContainerStarted","Data":"ca3df6498bb55931a928cc4411dd33ebde4932f135670e032d5b80b58e6f5fe4"}
Jul 10 20:08:35.459586 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.459550    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4" event={"ID":"f6365594-a556-40ad-bfbf-dcdb91f5a5d8","Type":"ContainerStarted","Data":"66c2a39b37157e2d7a80d151dcc257390188e47f04dee86bc0cc072b9df87b61"}
Jul 10 20:08:35.472099 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.472071    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:08:35.485944 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.485904    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" event={"ID":"b4dc4522d52b12e6a46164476b50912d","Type":"ContainerStarted","Data":"2c8ad3a81375709cfb1a32fa198d9958aa8738f1315a98bcbd73cfb12227540f"}
Jul 10 20:08:35.501773 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.501736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hpfkh" event={"ID":"222d6857-2959-42ad-b364-7322ba3bcb8e","Type":"ContainerStarted","Data":"de2cdd9cdfaca1fafcaccb42c6baf464f14012f3f8bcc8e74e7149ab70e0c536"}
Jul 10 20:08:35.827438 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.827353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:35.827438 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.827414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:35.827666 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.827584    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:35.827666 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.827644    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret podName:eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:37.827627052 +0000 UTC m=+5.053393963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret") pod "global-pull-secret-syncer-szwhd" (UID: "eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:35.828137 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.828043    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:35.828137 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.828097    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs podName:d9f1c3d3-65c9-4365-8dc8-0517ceae9708 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:37.828080483 +0000 UTC m=+5.053847404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs") pod "network-metrics-daemon-4wx2c" (UID: "d9f1c3d3-65c9-4365-8dc8-0517ceae9708") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:35.928464 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.928428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:35.928710 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.928643    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:08:35.928710 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.928680    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:08:35.928710 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.928695    2580 projected.go:196] Error preparing data for projected volume kube-api-access-kwtbx for pod openshift-network-diagnostics/network-check-target-lzt8q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:35.928878 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:35.928751    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx podName:67a7d692-17e3-4349-a6eb-78a4361a27ce nodeName:}" failed. No retries permitted until 2026-07-10 20:08:37.928733683 +0000 UTC m=+5.154500604 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwtbx" (UniqueName: "kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx") pod "network-check-target-lzt8q" (UID: "67a7d692-17e3-4349-a6eb-78a4361a27ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:35.947549 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:35.947519    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 20:08:36.156036 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:36.155913    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 20:03:34 +0000 UTC" deadline="2028-04-03 12:17:02.015267222 +0000 UTC"
Jul 10 20:08:36.156036 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:36.155953    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15184h8m25.859319082s"
Jul 10 20:08:36.275652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:36.275609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:36.275829 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:36.275776    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:36.276170 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:36.276151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:36.276317 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:36.276249    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:37.277183 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:37.277108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:37.277617 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.277242    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:37.848215 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:37.847701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:37.848215 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:37.847811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:37.848215 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.847965    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:37.848215 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.848027    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs podName:d9f1c3d3-65c9-4365-8dc8-0517ceae9708 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:41.848005928 +0000 UTC m=+9.073772840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs") pod "network-metrics-daemon-4wx2c" (UID: "d9f1c3d3-65c9-4365-8dc8-0517ceae9708") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:37.848215 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.848119    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:37.848215 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.848169    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret podName:eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:41.848157631 +0000 UTC m=+9.073924538 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret") pod "global-pull-secret-syncer-szwhd" (UID: "eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:37.948942 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:37.948907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:37.949131 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.949113    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:08:37.949204 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.949144    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:08:37.949204 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.949157    2580 projected.go:196] Error preparing data for projected volume kube-api-access-kwtbx for pod openshift-network-diagnostics/network-check-target-lzt8q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:37.949292 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:37.949214    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx podName:67a7d692-17e3-4349-a6eb-78a4361a27ce nodeName:}" failed. No retries permitted until 2026-07-10 20:08:41.949196204 +0000 UTC m=+9.174963126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwtbx" (UniqueName: "kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx") pod "network-check-target-lzt8q" (UID: "67a7d692-17e3-4349-a6eb-78a4361a27ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:38.275116 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:38.274916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:38.275116 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:38.274916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:38.275116 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:38.275046    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:38.275116 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:38.275099    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:39.279476 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:39.279446    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:39.280029 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:39.279576    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:40.275445 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:40.275410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:40.275445 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:40.275430    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:40.275652 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:40.275555    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:40.275727 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:40.275702    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:41.275472 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:41.275434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:41.275942 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.275584    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:41.882960 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:41.882922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:41.883138 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:41.883027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:41.883138 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.883083    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:41.883138 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.883130    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:41.883312 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.883154    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret podName:eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:49.883139574 +0000 UTC m=+17.108906486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret") pod "global-pull-secret-syncer-szwhd" (UID: "eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:41.883312 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.883179    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs podName:d9f1c3d3-65c9-4365-8dc8-0517ceae9708 nodeName:}" failed. No retries permitted until 2026-07-10 20:08:49.883164432 +0000 UTC m=+17.108931388 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs") pod "network-metrics-daemon-4wx2c" (UID: "d9f1c3d3-65c9-4365-8dc8-0517ceae9708") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:41.983655 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:41.983620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:41.983841 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.983820    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:08:41.983907 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.983847    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:08:41.983907 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.983862    2580 projected.go:196] Error preparing data for projected volume kube-api-access-kwtbx for pod openshift-network-diagnostics/network-check-target-lzt8q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:41.984002 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:41.983927    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx podName:67a7d692-17e3-4349-a6eb-78a4361a27ce nodeName:}" failed. No retries permitted until 2026-07-10 20:08:49.983907935 +0000 UTC m=+17.209674846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwtbx" (UniqueName: "kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx") pod "network-check-target-lzt8q" (UID: "67a7d692-17e3-4349-a6eb-78a4361a27ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:42.275638 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:42.275531    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:42.276132 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:42.275653    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:42.276132 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:42.275536    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:42.276132 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:42.276094    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:43.277288 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:43.276087    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:43.277288 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:43.276203    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:44.274986 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:44.274953    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:44.275184 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:44.274959    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:44.275184 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:44.275078    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:44.275184 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:44.275145    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:45.274936 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:45.274902    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:45.275405 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:45.275031    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:46.274909 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:46.274873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:46.275080 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:46.274996    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:46.275080 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:46.275052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:46.275432 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:46.275169    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:47.274916 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:47.274885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:47.275094 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:47.275005    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:48.274734 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:48.274703    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:48.274734 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:48.274725    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:48.274974 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:48.274817    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:48.275040 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:48.275010    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:49.275108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:49.275071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:49.275543 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:49.275198    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:49.944620 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:49.944584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:49.944854 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:49.944630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:49.944854 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:49.944751    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:49.944854 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:49.944777    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:49.944854 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:49.944811    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs podName:d9f1c3d3-65c9-4365-8dc8-0517ceae9708 nodeName:}" failed. No retries permitted until 2026-07-10 20:09:05.944796501 +0000 UTC m=+33.170563412 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs") pod "network-metrics-daemon-4wx2c" (UID: "d9f1c3d3-65c9-4365-8dc8-0517ceae9708") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:08:49.944854 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:49.944840    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret podName:eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64 nodeName:}" failed. No retries permitted until 2026-07-10 20:09:05.944821953 +0000 UTC m=+33.170588868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret") pod "global-pull-secret-syncer-szwhd" (UID: "eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:08:50.045380 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:50.045337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:50.045566 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:50.045547    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:08:50.045609 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:50.045574    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:08:50.045609 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:50.045589    2580 projected.go:196] Error preparing data for projected volume kube-api-access-kwtbx for pod openshift-network-diagnostics/network-check-target-lzt8q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:50.045692 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:50.045683    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx podName:67a7d692-17e3-4349-a6eb-78a4361a27ce nodeName:}" failed. No retries permitted until 2026-07-10 20:09:06.045649021 +0000 UTC m=+33.271415947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwtbx" (UniqueName: "kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx") pod "network-check-target-lzt8q" (UID: "67a7d692-17e3-4349-a6eb-78a4361a27ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:08:50.275780 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:50.275687    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:50.275780 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:50.275727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:50.276346 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:50.275840    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:50.276346 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:50.275925    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:51.275809 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:51.275777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:51.276280 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:51.275900    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:52.275133 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:52.275097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:52.275324 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:52.275210    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:52.275324 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:52.275256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:52.275437 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:52.275337    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:53.275823 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:53.275785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:53.276284 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:53.275901    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:54.275390 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:54.275357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:54.275581 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:54.275357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:54.275581 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:54.275464    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:54.275581 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:54.275530    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:55.275490 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.275184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:55.276072 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:55.275614    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:55.552902 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.552733    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:08:55.553240 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.553216    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f224662-2d68-4d3a-bb00-16fa779f6af4" containerID="f6359abfc34cf8f0ac27fa16c3c5a19812a29f1a3a8f590555e07e7e171c4937" exitCode=1
Jul 10 20:08:55.553306 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.553286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"77cd8e579d7f5e9bc9c8ea3f461a159e91fa0c3c95800e0b9473ee48bb635813"}
Jul 10 20:08:55.553381 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.553369    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"4402ec3a8df54f45e11f0f7cf6de7d20632b796524af40a0ab3896270a93dd71"}
Jul 10 20:08:55.553437 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.553401    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"9bf04aebd1d6eea94500700187c3e2b63ca50c00bf5efe62bd24e8f897bb7a4d"}
Jul 10 20:08:55.553437 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.553429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerDied","Data":"f6359abfc34cf8f0ac27fa16c3c5a19812a29f1a3a8f590555e07e7e171c4937"}
Jul 10 20:08:55.553527 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.553458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"c66130fea2db5f3e56fd36f0e033869aa497a02854ac78ccaa49859a26e6da44"}
Jul 10 20:08:55.554643 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.554625    2580 generic.go:356] "Generic (PLEG): container finished" podID="d5f9721c0a1f950164b033209b6a13a6" containerID="852a47fd5d2faa3c9d130599abcbeeeead6a8285e4d8747aa13c220da1ae44dc" exitCode=0
Jul 10 20:08:55.554732 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.554701    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" event={"ID":"d5f9721c0a1f950164b033209b6a13a6","Type":"ContainerDied","Data":"852a47fd5d2faa3c9d130599abcbeeeead6a8285e4d8747aa13c220da1ae44dc"}
Jul 10 20:08:55.556036 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.556018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cbbbn" event={"ID":"edfc10a5-c760-4fb3-8ed1-49e0bffd7f80","Type":"ContainerStarted","Data":"8881caa7a5d02518231f59ee9b2d98b953e8ad31f49f7ff9d9b7d04f8361e997"}
Jul 10 20:08:55.557404 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.557383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l9rxz" event={"ID":"d06df879-3247-42e7-a523-e15253ff8ede","Type":"ContainerStarted","Data":"72a4326badb0c858a2747864e0316eb0637a6c6de357484d0368045ddf693cac"}
Jul 10 20:08:55.558642 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.558618    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4" event={"ID":"f6365594-a556-40ad-bfbf-dcdb91f5a5d8","Type":"ContainerStarted","Data":"38c8f34dead8a4617eaab769facebd5fdd293eb65445b6ae864d1a5918c14025"}
Jul 10 20:08:55.560037 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.560011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" event={"ID":"b4dc4522d52b12e6a46164476b50912d","Type":"ContainerStarted","Data":"670ca254ba7dab357efa8cff7bf21e2b7c7c32350dd2d3e2d909c6b3fcece76f"}
Jul 10 20:08:55.561405 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.561284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hpfkh" event={"ID":"222d6857-2959-42ad-b364-7322ba3bcb8e","Type":"ContainerStarted","Data":"a652b268d251d8239f91e7ae4515a488c917cedb6b79bd2d84d68ac31d794300"}
Jul 10 20:08:55.562565 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.562544    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zzsd6" event={"ID":"d89da9d9-5f26-4757-90ee-cf3daed84967","Type":"ContainerStarted","Data":"004f389825a4d1da20a89372e438dc9ec1fcea91ea695c3bbfba440d0aff6de0"}
Jul 10 20:08:55.563934 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.563915    2580 generic.go:356] "Generic (PLEG): container finished" podID="8620a41e-4cb4-4a0b-b54d-ef177115f6b3" containerID="d8d045220e3c398cc6429c769b6f18108736bbf4f659497f81e4be0cab08881c" exitCode=0
Jul 10 20:08:55.564028 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.563991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerDied","Data":"d8d045220e3c398cc6429c769b6f18108736bbf4f659497f81e4be0cab08881c"}
Jul 10 20:08:55.567356 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.567338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g7kn9" event={"ID":"44056a08-41c4-4faf-a8d3-d66637bef925","Type":"ContainerStarted","Data":"21078ec92b1f3c4a1d59a75641e88f32cca10fc078996254963b7dd58b463cee"}
Jul 10 20:08:55.581846 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.581806    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g7kn9" podStartSLOduration=2.598988602 podStartE2EDuration="22.581792033s" totalImagesPullingTime="19.982803431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:55.581748987 +0000 UTC m=+22.807515912" watchObservedRunningTime="2026-07-10 20:08:55.581792033 +0000 UTC m=+22.807558962"
Jul 10 20:08:55.598557 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.597798    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l9rxz" podStartSLOduration=2.225959274 podStartE2EDuration="22.597779462s" totalImagesPullingTime="20.371820188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:55.597127366 +0000 UTC m=+22.822894294" watchObservedRunningTime="2026-07-10 20:08:55.597779462 +0000 UTC m=+22.823546394"
Jul 10 20:08:55.630551 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.630504    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cbbbn" podStartSLOduration=2.289934903 podStartE2EDuration="22.630488872s" totalImagesPullingTime="20.340553969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:55.6300875 +0000 UTC m=+22.855854430" watchObservedRunningTime="2026-07-10 20:08:55.630488872 +0000 UTC m=+22.856255792"
Jul 10 20:08:55.642610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.642536    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zzsd6" podStartSLOduration=6.422871291 podStartE2EDuration="22.64252105s" totalImagesPullingTime="16.219649759s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:55.642467879 +0000 UTC m=+22.868234809" watchObservedRunningTime="2026-07-10 20:08:55.64252105 +0000 UTC m=+22.868287979"
Jul 10 20:08:55.658336 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.658293    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hpfkh" podStartSLOduration=2.211425771 podStartE2EDuration="22.658280334s" totalImagesPullingTime="20.446854563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:55.657831019 +0000 UTC m=+22.883597948" watchObservedRunningTime="2026-07-10 20:08:55.658280334 +0000 UTC m=+22.884047262"
Jul 10 20:08:55.672103 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:55.672054    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-106.ec2.internal" podStartSLOduration=21.672035797 podStartE2EDuration="21.672035797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:55.671768579 +0000 UTC m=+22.897535509" watchObservedRunningTime="2026-07-10 20:08:55.672035797 +0000 UTC m=+22.897802730"
Jul 10 20:08:56.274808 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.274774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:56.274808 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.274791    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:56.275070 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:56.274903    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:56.275070 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:56.275017    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:56.574186 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.574104    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bclrh" event={"ID":"2dacc1cf-8421-46ab-9e02-8a3db6053886","Type":"ContainerStarted","Data":"f8e215e617a530adfdcc8965f3b2f14a60a6e156fea1c4460dc37f22823ef568"}
Jul 10 20:08:56.576735 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.576707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" event={"ID":"d5f9721c0a1f950164b033209b6a13a6","Type":"ContainerStarted","Data":"cb4a1207106bf41f99902bffc1c54e1e8f0cef47e701f390e0e15ad95e15b5cb"}
Jul 10 20:08:56.579839 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.579819    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:08:56.580257 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.580179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"a8fb8784c9f0d05fa5b0839e5283b7391824358cb6fe411ce56903b368dae1c8"}
Jul 10 20:08:56.599538 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.599476    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bclrh" podStartSLOduration=3.727851624 podStartE2EDuration="23.599458348s" totalImagesPullingTime="19.871606724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:56.598948254 +0000 UTC m=+23.824715186" watchObservedRunningTime="2026-07-10 20:08:56.599458348 +0000 UTC m=+23.825225278"
Jul 10 20:08:56.614100 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.614044    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-106.ec2.internal" podStartSLOduration=22.614025492 podStartE2EDuration="22.614025492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:08:56.613939433 +0000 UTC m=+23.839706362" watchObservedRunningTime="2026-07-10 20:08:56.614025492 +0000 UTC m=+23.839792421"
Jul 10 20:08:56.776714 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:56.776532    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 20:08:57.184942 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:57.184828    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T20:08:56.776709146Z","UUID":"a375cc02-1fb9-4ac1-b060-a3093028997d","Handler":null,"Name":"","Endpoint":""}
Jul 10 20:08:57.186483 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:57.186457    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 20:08:57.186621 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:57.186507    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 20:08:57.275117 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:57.275088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:57.275340 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:57.275221    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:08:57.367619 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:57.367587    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cbbbn_edfc10a5-c760-4fb3-8ed1-49e0bffd7f80/dns-node-resolver/0.log"
Jul 10 20:08:57.584560 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:57.584508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4" event={"ID":"f6365594-a556-40ad-bfbf-dcdb91f5a5d8","Type":"ContainerStarted","Data":"91f14627212ac5ae76553f165b5723d5746ed70cff9326fdd8199a95ad8e6d97"}
Jul 10 20:08:58.275079 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:58.275050    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:08:58.275272 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:58.275171    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:08:58.275272 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:58.275233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:08:58.275387 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:58.275346    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:08:58.590458 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:58.590386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:08:58.590982 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:58.590785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"102b4ccecf9c31306aec6d29ec19508f938749623ede8f29f49d2fec610ebe3c"}
Jul 10 20:08:58.745060 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:58.745033    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zzsd6_d89da9d9-5f26-4757-90ee-cf3daed84967/node-ca/0.log"
Jul 10 20:08:59.275160 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:08:59.275126    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:08:59.275723 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:08:59.275273    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:00.171150 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.170819    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:09:00.173654 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.172166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:09:00.275000 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.274968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:00.275163 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.274968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:00.275163 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:00.275088    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:00.275249 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:00.275166    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:00.597410 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.597366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4" event={"ID":"f6365594-a556-40ad-bfbf-dcdb91f5a5d8","Type":"ContainerStarted","Data":"922b690a0a874e5b39b62650d19d04ea28a41259f1682e4239e0d617fb17fe27"}
Jul 10 20:09:00.599090 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.599067    2580 generic.go:356] "Generic (PLEG): container finished" podID="8620a41e-4cb4-4a0b-b54d-ef177115f6b3" containerID="cb96dcbd3e27d548dba220b340587ca270d1d8b56da5fcba47020b5d34e86e2e" exitCode=0
Jul 10 20:09:00.599233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.599097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerDied","Data":"cb96dcbd3e27d548dba220b340587ca270d1d8b56da5fcba47020b5d34e86e2e"}
Jul 10 20:09:00.602535 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.602517    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:09:00.602871 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.602851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"a507b64efba136042db59b85a38eaf4c0b1cee28cc8360ac96ef76f5a09f11e9"}
Jul 10 20:09:00.603185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.603161    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:09:00.603369 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.603354    2580 scope.go:117] "RemoveContainer" containerID="f6359abfc34cf8f0ac27fa16c3c5a19812a29f1a3a8f590555e07e7e171c4937"
Jul 10 20:09:00.603552 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.603538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g7kn9"
Jul 10 20:09:00.616920 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:00.616881    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zrzr4" podStartSLOduration=2.865168253 podStartE2EDuration="27.616866227s" totalImagesPullingTime="24.751697974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:00.616862664 +0000 UTC m=+27.842629592" watchObservedRunningTime="2026-07-10 20:09:00.616866227 +0000 UTC m=+27.842633155"
Jul 10 20:09:01.274989 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.274959    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:01.275481 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:01.275085    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:01.608432 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.608366    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:09:01.608767 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.608738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" event={"ID":"6f224662-2d68-4d3a-bb00-16fa779f6af4","Type":"ContainerStarted","Data":"e9302e5453f45bf059cc13e2bb2af40d59fe08bdd05bbf915c5509dc5abe4715"}
Jul 10 20:09:01.608831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.608818    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 20:09:01.609065 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.609035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:09:01.609189 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.609117    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:09:01.624844 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.624818    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:09:01.624989 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.624929    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:09:01.639582 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:01.639535    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m" podStartSLOduration=8.218033833 podStartE2EDuration="28.639518687s" totalImagesPullingTime="20.421484854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:01.638373677 +0000 UTC m=+28.864140606" watchObservedRunningTime="2026-07-10 20:09:01.639518687 +0000 UTC m=+28.865285615"
Jul 10 20:09:02.275538 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:02.275498    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:02.275538 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:02.275509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:02.275989 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:02.275628    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:02.275989 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:02.275720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:02.613580 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:02.613501    2580 generic.go:356] "Generic (PLEG): container finished" podID="8620a41e-4cb4-4a0b-b54d-ef177115f6b3" containerID="51b6d6f284c98ac75c81a84621f659f229203388c5642f546a119992067c656e" exitCode=0
Jul 10 20:09:02.613748 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:02.613591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerDied","Data":"51b6d6f284c98ac75c81a84621f659f229203388c5642f546a119992067c656e"}
Jul 10 20:09:02.613748 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:02.613662    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 20:09:03.277090 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:03.276824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:03.277540 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:03.277184    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:03.567939 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:03.567851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:09:04.275166 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:04.275143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:04.275339 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:04.275143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:04.275339 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:04.275243    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:04.275339 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:04.275317    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:04.620431 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:04.620341    2580 generic.go:356] "Generic (PLEG): container finished" podID="8620a41e-4cb4-4a0b-b54d-ef177115f6b3" containerID="7dd92abbd7fe20a7265c784215730da91473b5ab28d31320369b0d6a78aa065b" exitCode=0
Jul 10 20:09:04.620856 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:04.620419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerDied","Data":"7dd92abbd7fe20a7265c784215730da91473b5ab28d31320369b0d6a78aa065b"}
Jul 10 20:09:05.275355 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:05.275319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:05.275545 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:05.275459    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:05.964944 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:05.964899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:05.965415 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:05.964956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:05.965415 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:05.965066    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 20:09:05.965415 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:05.965066    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:09:05.965415 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:05.965120    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret podName:eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64 nodeName:}" failed. No retries permitted until 2026-07-10 20:09:37.965101818 +0000 UTC m=+65.190868738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret") pod "global-pull-secret-syncer-szwhd" (UID: "eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64") : object "kube-system"/"original-pull-secret" not registered
Jul 10 20:09:05.965415 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:05.965138    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs podName:d9f1c3d3-65c9-4365-8dc8-0517ceae9708 nodeName:}" failed. No retries permitted until 2026-07-10 20:09:37.965128205 +0000 UTC m=+65.190895112 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs") pod "network-metrics-daemon-4wx2c" (UID: "d9f1c3d3-65c9-4365-8dc8-0517ceae9708") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 20:09:06.065321 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:06.065283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:06.065499 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:06.065418    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 20:09:06.065499 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:06.065432    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 20:09:06.065499 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:06.065442    2580 projected.go:196] Error preparing data for projected volume kube-api-access-kwtbx for pod openshift-network-diagnostics/network-check-target-lzt8q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:09:06.065499 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:06.065498    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx podName:67a7d692-17e3-4349-a6eb-78a4361a27ce nodeName:}" failed. No retries permitted until 2026-07-10 20:09:38.065480774 +0000 UTC m=+65.291247688 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwtbx" (UniqueName: "kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx") pod "network-check-target-lzt8q" (UID: "67a7d692-17e3-4349-a6eb-78a4361a27ce") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 20:09:06.275374 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:06.275342    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:06.275538 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:06.275349    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:06.275538 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:06.275474    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:06.275642 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:06.275549    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:07.275326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:07.275292    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:07.275917 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:07.275418    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:08.275317 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:08.275283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:08.275317 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:08.275326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:08.275817 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:08.275424    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:08.275817 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:08.275559    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:09.275080 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:09.275052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:09.275235 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:09.275161    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:09.635892 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:09.635861    2580 generic.go:356] "Generic (PLEG): container finished" podID="8620a41e-4cb4-4a0b-b54d-ef177115f6b3" containerID="65728af440ef44f9fd90ba8ff1f2939ae2e1c23ba9eddd723cd3f1feb04f21c6" exitCode=0
Jul 10 20:09:09.636256 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:09.635920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerDied","Data":"65728af440ef44f9fd90ba8ff1f2939ae2e1c23ba9eddd723cd3f1feb04f21c6"}
Jul 10 20:09:10.275211 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:10.275182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:10.275372 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:10.275182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:10.275372 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:10.275316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:10.275372 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:10.275352    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:10.641060 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:10.640978    2580 generic.go:356] "Generic (PLEG): container finished" podID="8620a41e-4cb4-4a0b-b54d-ef177115f6b3" containerID="77ac083af12664e30a0fe2d3a10f3e1ca2106bc5eccbe8ee851963e4abfd4597" exitCode=0
Jul 10 20:09:10.641060 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:10.641027    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerDied","Data":"77ac083af12664e30a0fe2d3a10f3e1ca2106bc5eccbe8ee851963e4abfd4597"}
Jul 10 20:09:11.275508 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:11.275471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:11.275704 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:11.275605    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:11.646302 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:11.646216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" event={"ID":"8620a41e-4cb4-4a0b-b54d-ef177115f6b3","Type":"ContainerStarted","Data":"fcc9685e5672831251da77752094d0d376c53412d5861ad5f606216cc45c1b46"}
Jul 10 20:09:11.669885 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:11.669839    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nzdm9" podStartSLOduration=7.445136468 podStartE2EDuration="38.669824851s" totalImagesPullingTime="31.224688383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:11.669478202 +0000 UTC m=+38.895245131" watchObservedRunningTime="2026-07-10 20:09:11.669824851 +0000 UTC m=+38.895591779"
Jul 10 20:09:12.275684 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:12.275642    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:12.275854 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:12.275642    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:12.275854 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:12.275766    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:12.275854 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:12.275839    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:13.276070 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:13.276036    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:13.276577 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:13.276130    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:14.274798 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:14.274747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:14.274986 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:14.274744    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:14.274986 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:14.274876    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:14.274986 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:14.274931    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:15.275678 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:15.275637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:15.276036 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:15.275760    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:16.275231 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:16.275174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:16.275231 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:16.275225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:16.275486 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:16.275320    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:16.275486 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:16.275428    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:17.275168 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:17.275137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:17.275612 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:17.275271    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:18.275036 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:18.275001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:18.275323 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:18.275123    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:18.275323 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:18.275167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:18.275323 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:18.275261    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:19.275739 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:19.275454    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:19.276177 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:19.275847    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:19.407145 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:19.407106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-szwhd"]
Jul 10 20:09:19.410539 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:19.410509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lzt8q"]
Jul 10 20:09:19.410688 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:19.410636    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:19.410910 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:19.410759    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:19.411300 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:19.411276    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4wx2c"]
Jul 10 20:09:19.411427 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:19.411412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:19.411557 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:19.411531    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:19.667236 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:19.667204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:19.667376 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:19.667332    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:21.275366 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:21.275334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:21.275366 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:21.275353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:21.275798 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:21.275340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:21.275798 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:21.275474    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:21.275798 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:21.275552    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:21.275798 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:21.275639    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:23.276327 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.276291    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:23.276714 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.276369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:23.276714 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:23.276386    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lzt8q" podUID="67a7d692-17e3-4349-a6eb-78a4361a27ce"
Jul 10 20:09:23.276714 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:23.276481    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4wx2c" podUID="d9f1c3d3-65c9-4365-8dc8-0517ceae9708"
Jul 10 20:09:23.276714 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.276509    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:23.276714 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:23.276567    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-szwhd" podUID="eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64"
Jul 10 20:09:23.623004 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.622927    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-106.ec2.internal" event="NodeReady"
Jul 10 20:09:23.623163 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.623066    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 20:09:23.655857 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.655829    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7898cfcbf4-62vs7"]
Jul 10 20:09:23.668013 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.667987    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.669921 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.669885    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7898cfcbf4-62vs7"]
Jul 10 20:09:23.670538 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.670511    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 20:09:23.670538 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.670532    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 20:09:23.670698 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.670518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5wwzb\""
Jul 10 20:09:23.670698 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.670659    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 20:09:23.673836 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.673286    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fl6wl"]
Jul 10 20:09:23.675825 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.675808    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 20:09:23.682834 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.682816    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-89fvm"]
Jul 10 20:09:23.682949 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.682934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:23.685405 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.685384    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 20:09:23.685520 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.685390    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 20:09:23.685520 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.685425    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 20:09:23.685520 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.685425    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-n22ht\""
Jul 10 20:09:23.696750 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.696730    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fl6wl"]
Jul 10 20:09:23.696869 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.696759    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89fvm"]
Jul 10 20:09:23.696869 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.696862    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.699722 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.699700    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 20:09:23.699976 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.699953    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 20:09:23.700209 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.700193    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fsg7c\""
Jul 10 20:09:23.701994 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.701970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-registry-tls\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.702191 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76fad17c-35dc-4aa6-b566-cfb59a318a33-registry-certificates\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.702281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjsq4\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-kube-api-access-pjsq4\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.702281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39306aff-c570-417f-9d22-7db335d5909e-cert\") pod \"ingress-canary-fl6wl\" (UID: \"39306aff-c570-417f-9d22-7db335d5909e\") " pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:23.702281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0236af-836d-44dc-ab84-dbfba301a7f8-tmp-dir\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.702404 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx882\" (UniqueName: \"kubernetes.io/projected/2d0236af-836d-44dc-ab84-dbfba301a7f8-kube-api-access-jx882\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.702447 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76fad17c-35dc-4aa6-b566-cfb59a318a33-installation-pull-secrets\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.702489 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d0236af-836d-44dc-ab84-dbfba301a7f8-metrics-tls\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.702536 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76fad17c-35dc-4aa6-b566-cfb59a318a33-ca-trust-extracted\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.702849 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76fad17c-35dc-4aa6-b566-cfb59a318a33-trusted-ca\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.702908 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr99k\" (UniqueName: \"kubernetes.io/projected/39306aff-c570-417f-9d22-7db335d5909e-kube-api-access-tr99k\") pod \"ingress-canary-fl6wl\" (UID: \"39306aff-c570-417f-9d22-7db335d5909e\") " pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:23.702908 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76fad17c-35dc-4aa6-b566-cfb59a318a33-image-registry-private-configuration\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.702908 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0236af-836d-44dc-ab84-dbfba301a7f8-config-volume\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.703046 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.702975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-bound-sa-token\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.774810 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.774771    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vds78"]
Jul 10 20:09:23.789760 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.789731    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vds78"]
Jul 10 20:09:23.789907 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.789849    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.792242 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.792218    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 20:09:23.792369 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.792271    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 20:09:23.792441 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.792378    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 20:09:23.792495 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.792459    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 20:09:23.792725 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.792705    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-djccr\""
Jul 10 20:09:23.804105 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76fad17c-35dc-4aa6-b566-cfb59a318a33-installation-pull-secrets\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a845fc09-5764-45bb-9a1f-2270ed43222a-crio-socket\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.804216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d0236af-836d-44dc-ab84-dbfba301a7f8-metrics-tls\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.804216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59kf4\" (UniqueName: \"kubernetes.io/projected/a845fc09-5764-45bb-9a1f-2270ed43222a-kube-api-access-59kf4\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.804216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76fad17c-35dc-4aa6-b566-cfb59a318a33-ca-trust-extracted\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76fad17c-35dc-4aa6-b566-cfb59a318a33-trusted-ca\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr99k\" (UniqueName: \"kubernetes.io/projected/39306aff-c570-417f-9d22-7db335d5909e-kube-api-access-tr99k\") pod \"ingress-canary-fl6wl\" (UID: \"39306aff-c570-417f-9d22-7db335d5909e\") " pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:23.804416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76fad17c-35dc-4aa6-b566-cfb59a318a33-image-registry-private-configuration\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804550 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0236af-836d-44dc-ab84-dbfba301a7f8-config-volume\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.804550 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a845fc09-5764-45bb-9a1f-2270ed43222a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.804550 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a845fc09-5764-45bb-9a1f-2270ed43222a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.804550 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-bound-sa-token\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804550 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804535    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-registry-tls\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a845fc09-5764-45bb-9a1f-2270ed43222a-data-volume\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.804815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76fad17c-35dc-4aa6-b566-cfb59a318a33-registry-certificates\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjsq4\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-kube-api-access-pjsq4\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.804815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39306aff-c570-417f-9d22-7db335d5909e-cert\") pod \"ingress-canary-fl6wl\" (UID: \"39306aff-c570-417f-9d22-7db335d5909e\") " pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:23.804815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804642    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/76fad17c-35dc-4aa6-b566-cfb59a318a33-ca-trust-extracted\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.805335 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.805310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/76fad17c-35dc-4aa6-b566-cfb59a318a33-trusted-ca\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.805335 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.805330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d0236af-836d-44dc-ab84-dbfba301a7f8-config-volume\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.805484 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.804644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0236af-836d-44dc-ab84-dbfba301a7f8-tmp-dir\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.805484 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.805371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx882\" (UniqueName: \"kubernetes.io/projected/2d0236af-836d-44dc-ab84-dbfba301a7f8-kube-api-access-jx882\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.805839 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.805821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d0236af-836d-44dc-ab84-dbfba301a7f8-tmp-dir\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.806016 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.805996    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/76fad17c-35dc-4aa6-b566-cfb59a318a33-registry-certificates\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.808729 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.808703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/76fad17c-35dc-4aa6-b566-cfb59a318a33-image-registry-private-configuration\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.808826 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.808729    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d0236af-836d-44dc-ab84-dbfba301a7f8-metrics-tls\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.808826 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.808736    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/76fad17c-35dc-4aa6-b566-cfb59a318a33-installation-pull-secrets\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.808826 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.808768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-registry-tls\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.809021 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.808847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39306aff-c570-417f-9d22-7db335d5909e-cert\") pod \"ingress-canary-fl6wl\" (UID: \"39306aff-c570-417f-9d22-7db335d5909e\") " pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:23.813038 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.813020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx882\" (UniqueName: \"kubernetes.io/projected/2d0236af-836d-44dc-ab84-dbfba301a7f8-kube-api-access-jx882\") pod \"dns-default-89fvm\" (UID: \"2d0236af-836d-44dc-ab84-dbfba301a7f8\") " pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:23.813378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.813360    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr99k\" (UniqueName: \"kubernetes.io/projected/39306aff-c570-417f-9d22-7db335d5909e-kube-api-access-tr99k\") pod \"ingress-canary-fl6wl\" (UID: \"39306aff-c570-417f-9d22-7db335d5909e\") " pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:23.813473 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.813456    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjsq4\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-kube-api-access-pjsq4\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.813594 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.813579    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/76fad17c-35dc-4aa6-b566-cfb59a318a33-bound-sa-token\") pod \"image-registry-7898cfcbf4-62vs7\" (UID: \"76fad17c-35dc-4aa6-b566-cfb59a318a33\") " pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.905964 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.905874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a845fc09-5764-45bb-9a1f-2270ed43222a-crio-socket\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.905964 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.905916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59kf4\" (UniqueName: \"kubernetes.io/projected/a845fc09-5764-45bb-9a1f-2270ed43222a-kube-api-access-59kf4\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.906183 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.905968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a845fc09-5764-45bb-9a1f-2270ed43222a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.906183 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.905985    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a845fc09-5764-45bb-9a1f-2270ed43222a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.906183 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.906025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a845fc09-5764-45bb-9a1f-2270ed43222a-data-volume\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.906183 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.906085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a845fc09-5764-45bb-9a1f-2270ed43222a-crio-socket\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.906383 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.906363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a845fc09-5764-45bb-9a1f-2270ed43222a-data-volume\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.906564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.906537    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a845fc09-5764-45bb-9a1f-2270ed43222a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.908275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.908255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a845fc09-5764-45bb-9a1f-2270ed43222a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.913343 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.913321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59kf4\" (UniqueName: \"kubernetes.io/projected/a845fc09-5764-45bb-9a1f-2270ed43222a-kube-api-access-59kf4\") pod \"insights-runtime-extractor-vds78\" (UID: \"a845fc09-5764-45bb-9a1f-2270ed43222a\") " pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:23.979745 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.979711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:23.992857 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:23.992833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fl6wl"
Jul 10 20:09:24.007164 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.007134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:24.099550 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.099523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vds78"
Jul 10 20:09:24.195242 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.194683    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fl6wl"]
Jul 10 20:09:24.201046 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.200106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7898cfcbf4-62vs7"]
Jul 10 20:09:24.218177 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:24.218142    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39306aff_c570_417f_9d22_7db335d5909e.slice/crio-c3a8601963238dac7977ecfbd7f4ac9d4b7ca56857809cba055a6c83e04fde43 WatchSource:0}: Error finding container c3a8601963238dac7977ecfbd7f4ac9d4b7ca56857809cba055a6c83e04fde43: Status 404 returned error can't find the container with id c3a8601963238dac7977ecfbd7f4ac9d4b7ca56857809cba055a6c83e04fde43
Jul 10 20:09:24.218787 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.218703    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89fvm"]
Jul 10 20:09:24.219554 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:24.219531    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d0236af_836d_44dc_ab84_dbfba301a7f8.slice/crio-f7a663033892c82ce9404a1f1124d50b599622e0d1a3533019336eabf595f5c9 WatchSource:0}: Error finding container f7a663033892c82ce9404a1f1124d50b599622e0d1a3533019336eabf595f5c9: Status 404 returned error can't find the container with id f7a663033892c82ce9404a1f1124d50b599622e0d1a3533019336eabf595f5c9
Jul 10 20:09:24.282540 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.282509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vds78"]
Jul 10 20:09:24.285036 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:24.285010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda845fc09_5764_45bb_9a1f_2270ed43222a.slice/crio-885830737a569bce2b58a4c5122c5d0d3a5656568c22114faa20980b209a7351 WatchSource:0}: Error finding container 885830737a569bce2b58a4c5122c5d0d3a5656568c22114faa20980b209a7351: Status 404 returned error can't find the container with id 885830737a569bce2b58a4c5122c5d0d3a5656568c22114faa20980b209a7351
Jul 10 20:09:24.679036 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.678998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vds78" event={"ID":"a845fc09-5764-45bb-9a1f-2270ed43222a","Type":"ContainerStarted","Data":"885830737a569bce2b58a4c5122c5d0d3a5656568c22114faa20980b209a7351"}
Jul 10 20:09:24.680109 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.680087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89fvm" event={"ID":"2d0236af-836d-44dc-ab84-dbfba301a7f8","Type":"ContainerStarted","Data":"f7a663033892c82ce9404a1f1124d50b599622e0d1a3533019336eabf595f5c9"}
Jul 10 20:09:24.681156 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.681132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fl6wl" event={"ID":"39306aff-c570-417f-9d22-7db335d5909e","Type":"ContainerStarted","Data":"c3a8601963238dac7977ecfbd7f4ac9d4b7ca56857809cba055a6c83e04fde43"}
Jul 10 20:09:24.682484 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.682459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7" event={"ID":"76fad17c-35dc-4aa6-b566-cfb59a318a33","Type":"ContainerStarted","Data":"8507a6626416fc865f5a91683c762c6c2218feec9aa611f3f32f280fcfbaf401"}
Jul 10 20:09:24.682566 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.682498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7" event={"ID":"76fad17c-35dc-4aa6-b566-cfb59a318a33","Type":"ContainerStarted","Data":"aa86f2b674f85e8da387a87822a764853ca91437cb251fec22b631f850ccb179"}
Jul 10 20:09:24.682615 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.682600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:24.702340 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:24.702294    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7" podStartSLOduration=6.702276794 podStartE2EDuration="6.702276794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:24.70102686 +0000 UTC m=+51.926793790" watchObservedRunningTime="2026-07-10 20:09:24.702276794 +0000 UTC m=+51.928043700"
Jul 10 20:09:25.276723 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.275089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:25.276723 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.275507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:25.276723 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.276182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:25.280281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.279455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 20:09:25.280281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.279718    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 20:09:25.280281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.279966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 20:09:25.284363 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.281494    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 20:09:25.284363 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.282715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2llrd\""
Jul 10 20:09:25.285517 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:25.285433    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6l2jv\""
Jul 10 20:09:30.703971 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:30.703928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89fvm" event={"ID":"2d0236af-836d-44dc-ab84-dbfba301a7f8","Type":"ContainerStarted","Data":"2dec80ef38f2c45d9c74e8e02434e24ef5bd316f78870ae37a7704a266c0e8ad"}
Jul 10 20:09:30.705619 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:30.705589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fl6wl" event={"ID":"39306aff-c570-417f-9d22-7db335d5909e","Type":"ContainerStarted","Data":"dcda46c7144126a102c775fe19876dba63491d340ee81de6b23fc53e329ec627"}
Jul 10 20:09:30.721334 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:30.721284    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fl6wl" podStartSLOduration=1.44480375 podStartE2EDuration="7.721268723s" totalImagesPullingTime="6.276464973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:30.720171784 +0000 UTC m=+57.945938712" watchObservedRunningTime="2026-07-10 20:09:30.721268723 +0000 UTC m=+57.947035652"
Jul 10 20:09:31.712071 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:31.712033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vds78" event={"ID":"a845fc09-5764-45bb-9a1f-2270ed43222a","Type":"ContainerStarted","Data":"ee079d17eaf5020efe45617cd8af8ea5043438f35a25a2898b8e1ac769c08016"}
Jul 10 20:09:31.713994 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:31.713965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89fvm" event={"ID":"2d0236af-836d-44dc-ab84-dbfba301a7f8","Type":"ContainerStarted","Data":"cccd7c0037516c521cd24f67209165278bce92a8146bcb13b2f7de9a685272c5"}
Jul 10 20:09:31.730129 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:31.730075    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-89fvm" podStartSLOduration=2.46529455 podStartE2EDuration="8.730058437s" totalImagesPullingTime="6.264763887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:31.729434696 +0000 UTC m=+58.955201624" watchObservedRunningTime="2026-07-10 20:09:31.730058437 +0000 UTC m=+58.955825365"
Jul 10 20:09:32.719358 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:32.719320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vds78" event={"ID":"a845fc09-5764-45bb-9a1f-2270ed43222a","Type":"ContainerStarted","Data":"175a368fbb1c4cb1f219d798f57af9fd9c500ae1a99f21dfcc7e537df5c91bd0"}
Jul 10 20:09:32.719861 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:32.719560    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:34.631145 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:34.631117    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dqm9m"
Jul 10 20:09:34.727961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:34.727919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vds78" event={"ID":"a845fc09-5764-45bb-9a1f-2270ed43222a","Type":"ContainerStarted","Data":"4e16bae3a5d54acaa86bad88a62d0c88f5bfde93f6678fa7cd6b5ff8efbb7a03"}
Jul 10 20:09:34.746315 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:34.746258    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vds78" podStartSLOduration=1.969148057 podStartE2EDuration="11.746239173s" totalImagesPullingTime="9.777091116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:09:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:34.74558527 +0000 UTC m=+61.971352198" watchObservedRunningTime="2026-07-10 20:09:34.746239173 +0000 UTC m=+61.972006102"
Jul 10 20:09:36.877864 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.877830    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-w8gwm"]
Jul 10 20:09:36.879739 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.879724    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:36.882047 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.882025    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 10 20:09:36.882047 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.882025    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 10 20:09:36.882239 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.882118    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kg4l7\""
Jul 10 20:09:36.883190 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.883172    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 20:09:36.883371 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.883209    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 20:09:36.883371 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.883220    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 20:09:36.888280 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.888254    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-w8gwm"]
Jul 10 20:09:36.916588 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.916549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6w7d\" (UniqueName: \"kubernetes.io/projected/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-kube-api-access-z6w7d\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:36.916588 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.916588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:36.916812 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.916614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-metrics-client-ca\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:36.916812 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:36.916757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.017865 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.017822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.017865 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.017863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-metrics-client-ca\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.018119 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.017921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.018119 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.017954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6w7d\" (UniqueName: \"kubernetes.io/projected/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-kube-api-access-z6w7d\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.018119 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:37.017986    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 10 20:09:37.018119 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:37.018059    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-tls podName:c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220 nodeName:}" failed. No retries permitted until 2026-07-10 20:09:37.518041069 +0000 UTC m=+64.743807979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-tls") pod "prometheus-operator-64785b6675-w8gwm" (UID: "c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220") : secret "prometheus-operator-tls" not found
Jul 10 20:09:37.018659 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.018635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-metrics-client-ca\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.021127 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.021109    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.026145 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.026115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6w7d\" (UniqueName: \"kubernetes.io/projected/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-kube-api-access-z6w7d\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.521549 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.521503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.523875 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.523822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-w8gwm\" (UID: \"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220\") " pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.789009 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.788907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm"
Jul 10 20:09:37.912982 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:37.912952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-w8gwm"]
Jul 10 20:09:37.916166 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:37.916134    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d6a5e6_2c93_4b16_a6fb_80b68fe8d220.slice/crio-de06e658489bd287fb9a9cae82e83f8761cbb6ec1ce8918c2830ebb94296774a WatchSource:0}: Error finding container de06e658489bd287fb9a9cae82e83f8761cbb6ec1ce8918c2830ebb94296774a: Status 404 returned error can't find the container with id de06e658489bd287fb9a9cae82e83f8761cbb6ec1ce8918c2830ebb94296774a
Jul 10 20:09:38.024387 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.024352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:38.024548 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.024408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:38.027076 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.027052    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 20:09:38.027212 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.027084    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 20:09:38.037314 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.037283    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9f1c3d3-65c9-4365-8dc8-0517ceae9708-metrics-certs\") pod \"network-metrics-daemon-4wx2c\" (UID: \"d9f1c3d3-65c9-4365-8dc8-0517ceae9708\") " pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:38.037404 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.037368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64-original-pull-secret\") pod \"global-pull-secret-syncer-szwhd\" (UID: \"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64\") " pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:38.125770 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.125683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:38.128405 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.128386    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 20:09:38.138734 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.138711    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 20:09:38.148940 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.148920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwtbx\" (UniqueName: \"kubernetes.io/projected/67a7d692-17e3-4349-a6eb-78a4361a27ce-kube-api-access-kwtbx\") pod \"network-check-target-lzt8q\" (UID: \"67a7d692-17e3-4349-a6eb-78a4361a27ce\") " pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:38.197875 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.197845    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2llrd\""
Jul 10 20:09:38.205884 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.205863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4wx2c"
Jul 10 20:09:38.208142 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.208130    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6l2jv\""
Jul 10 20:09:38.215120 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.215096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-szwhd"
Jul 10 20:09:38.216216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.216186    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:38.342661 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.342628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4wx2c"]
Jul 10 20:09:38.346882 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:38.346833    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9f1c3d3_65c9_4365_8dc8_0517ceae9708.slice/crio-9138271f75f196a75287b848cc084becd7549afd42373d5f9a3103ad9069eec7 WatchSource:0}: Error finding container 9138271f75f196a75287b848cc084becd7549afd42373d5f9a3103ad9069eec7: Status 404 returned error can't find the container with id 9138271f75f196a75287b848cc084becd7549afd42373d5f9a3103ad9069eec7
Jul 10 20:09:38.360654 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.360617    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-szwhd"]
Jul 10 20:09:38.363560 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:38.363538    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9b3be5_7e24_4e37_9f4b_f7bf7d8cab64.slice/crio-473fa1d3af552b6729a25f0558360eb17eca33e5599c81fc87c445708734dc08 WatchSource:0}: Error finding container 473fa1d3af552b6729a25f0558360eb17eca33e5599c81fc87c445708734dc08: Status 404 returned error can't find the container with id 473fa1d3af552b6729a25f0558360eb17eca33e5599c81fc87c445708734dc08
Jul 10 20:09:38.375251 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.375227    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lzt8q"]
Jul 10 20:09:38.379689 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:38.379651    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a7d692_17e3_4349_a6eb_78a4361a27ce.slice/crio-f7f257dd560856b6c30e7cd318d4e0b44cfd5a5fde6e99f645b5e0c5308dc27e WatchSource:0}: Error finding container f7f257dd560856b6c30e7cd318d4e0b44cfd5a5fde6e99f645b5e0c5308dc27e: Status 404 returned error can't find the container with id f7f257dd560856b6c30e7cd318d4e0b44cfd5a5fde6e99f645b5e0c5308dc27e
Jul 10 20:09:38.741133 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.741048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wx2c" event={"ID":"d9f1c3d3-65c9-4365-8dc8-0517ceae9708","Type":"ContainerStarted","Data":"9138271f75f196a75287b848cc084becd7549afd42373d5f9a3103ad9069eec7"}
Jul 10 20:09:38.742276 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.742245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lzt8q" event={"ID":"67a7d692-17e3-4349-a6eb-78a4361a27ce","Type":"ContainerStarted","Data":"f7f257dd560856b6c30e7cd318d4e0b44cfd5a5fde6e99f645b5e0c5308dc27e"}
Jul 10 20:09:38.743345 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.743314    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-szwhd" event={"ID":"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64","Type":"ContainerStarted","Data":"473fa1d3af552b6729a25f0558360eb17eca33e5599c81fc87c445708734dc08"}
Jul 10 20:09:38.744443 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:38.744421    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm" event={"ID":"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220","Type":"ContainerStarted","Data":"de06e658489bd287fb9a9cae82e83f8761cbb6ec1ce8918c2830ebb94296774a"}
Jul 10 20:09:39.756861 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:39.756774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm" event={"ID":"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220","Type":"ContainerStarted","Data":"8b01606f276d200b3d00360ed1b980cc9311c03e909539f7a3010cf99799e954"}
Jul 10 20:09:40.763230 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:40.763183    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm" event={"ID":"c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220","Type":"ContainerStarted","Data":"7bdd3ad3e5e3c484d183408827eedefadb320c99c59dfbc94dbdc4213aed1a42"}
Jul 10 20:09:40.765151 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:40.765124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wx2c" event={"ID":"d9f1c3d3-65c9-4365-8dc8-0517ceae9708","Type":"ContainerStarted","Data":"68eda79098557517a6cd596c537a3308301014a49fad1f852f09a853a4148a38"}
Jul 10 20:09:40.765285 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:40.765174    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4wx2c" event={"ID":"d9f1c3d3-65c9-4365-8dc8-0517ceae9708","Type":"ContainerStarted","Data":"ee13940a6e8eba305699f8f979eb407f2b993b761ba13547e1e68484b9cac91d"}
Jul 10 20:09:40.796767 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:40.796713    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4wx2c" podStartSLOduration=66.333245137 podStartE2EDuration="1m7.796695541s" totalImagesPullingTime="1.463450404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:40.796021221 +0000 UTC m=+68.021788153" watchObservedRunningTime="2026-07-10 20:09:40.796695541 +0000 UTC m=+68.022462470"
Jul 10 20:09:40.797218 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:40.797171    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-w8gwm" podStartSLOduration=3.297480763 podStartE2EDuration="4.797155331s" totalImagesPullingTime="1.499674568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:40.780697838 +0000 UTC m=+68.006464768" watchObservedRunningTime="2026-07-10 20:09:40.797155331 +0000 UTC m=+68.022922262"
Jul 10 20:09:42.252205 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.252169    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5v6j2"]
Jul 10 20:09:42.274641 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.274595    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-t2c72"]
Jul 10 20:09:42.274825 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.274762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.277296 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.277270    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 20:09:42.277429 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.277296    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-twgm6\""
Jul 10 20:09:42.277551 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.277531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 20:09:42.277687 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.277554    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 20:09:42.288476 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.288451    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-t2c72"]
Jul 10 20:09:42.288593 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.288580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.291196 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.291036    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 10 20:09:42.291196 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.291037    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 10 20:09:42.291548 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.291527    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 10 20:09:42.291548 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.291537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-f9nzt\""
Jul 10 20:09:42.365027 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.364991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.365219 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rxzn\" (UniqueName: \"kubernetes.io/projected/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-api-access-7rxzn\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.365219 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5q77\" (UniqueName: \"kubernetes.io/projected/e68fbbfc-8e68-49fa-8725-9efb03195902-kube-api-access-t5q77\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365219 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-tls\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365219 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365195    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-sys\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e68fbbfc-8e68-49fa-8725-9efb03195902-metrics-client-ca\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c4bc253-6422-4be4-83d4-6bb829538a24-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.365435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.365435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-textfile\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-root\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365712 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.365712 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-wtmp\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365712 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365712 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-accelerators-collector-config\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.365712 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.365537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c4bc253-6422-4be4-83d4-6bb829538a24-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.466387 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-textfile\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.466387 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-root\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-wtmp\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-root\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-accelerators-collector-config\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c4bc253-6422-4be4-83d4-6bb829538a24-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:42.466563    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rxzn\" (UniqueName: \"kubernetes.io/projected/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-api-access-7rxzn\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.466635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5q77\" (UniqueName: \"kubernetes.io/projected/e68fbbfc-8e68-49fa-8725-9efb03195902-kube-api-access-t5q77\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-tls\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:42.466748    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls podName:2c4bc253-6422-4be4-83d4-6bb829538a24 nodeName:}" failed. No retries permitted until 2026-07-10 20:09:42.966709894 +0000 UTC m=+70.192476803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-t2c72" (UID: "2c4bc253-6422-4be4-83d4-6bb829538a24") : secret "kube-state-metrics-tls" not found
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-textfile\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-sys\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e68fbbfc-8e68-49fa-8725-9efb03195902-metrics-client-ca\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c4bc253-6422-4be4-83d4-6bb829538a24-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.467002    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2c4bc253-6422-4be4-83d4-6bb829538a24-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.467112 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.466631    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-wtmp\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467582 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.467144    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e68fbbfc-8e68-49fa-8725-9efb03195902-sys\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467582 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.467544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-accelerators-collector-config\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.467712 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.467639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.467883 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.467789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e68fbbfc-8e68-49fa-8725-9efb03195902-metrics-client-ca\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.468003 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.467979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2c4bc253-6422-4be4-83d4-6bb829538a24-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.469455 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.469434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-tls\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.469758 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.469741    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.470001 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.469972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e68fbbfc-8e68-49fa-8725-9efb03195902-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.476025 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.475980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rxzn\" (UniqueName: \"kubernetes.io/projected/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-api-access-7rxzn\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.479125 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.479106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5q77\" (UniqueName: \"kubernetes.io/projected/e68fbbfc-8e68-49fa-8725-9efb03195902-kube-api-access-t5q77\") pod \"node-exporter-5v6j2\" (UID: \"e68fbbfc-8e68-49fa-8725-9efb03195902\") " pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.586658 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.586619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5v6j2"
Jul 10 20:09:42.725102 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.725069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-89fvm"
Jul 10 20:09:42.972616 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:42.972523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:42.972797 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:42.972690    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 10 20:09:42.972797 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:09:42.972773    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls podName:2c4bc253-6422-4be4-83d4-6bb829538a24 nodeName:}" failed. No retries permitted until 2026-07-10 20:09:43.972750592 +0000 UTC m=+71.198517503 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-t2c72" (UID: "2c4bc253-6422-4be4-83d4-6bb829538a24") : secret "kube-state-metrics-tls" not found
Jul 10 20:09:43.088425 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:43.088390    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68fbbfc_8e68_49fa_8725_9efb03195902.slice/crio-0cd80e6ae5e54e6a81648d3b6b6513571aaba2c37aa8c5e6ab2a3841493c86fd WatchSource:0}: Error finding container 0cd80e6ae5e54e6a81648d3b6b6513571aaba2c37aa8c5e6ab2a3841493c86fd: Status 404 returned error can't find the container with id 0cd80e6ae5e54e6a81648d3b6b6513571aaba2c37aa8c5e6ab2a3841493c86fd
Jul 10 20:09:43.265660 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.265587    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:09:43.283897 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.283858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.286796 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.286541    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 10 20:09:43.286796 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.286732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 10 20:09:43.287119 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.286901    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 10 20:09:43.287554 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.287284    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 10 20:09:43.287554 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.287394    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 10 20:09:43.287554 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.287414    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dkvl8\""
Jul 10 20:09:43.287554 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.287420    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 10 20:09:43.287554 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.287502    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 10 20:09:43.288077 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.287561    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 10 20:09:43.288077 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.287701    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 10 20:09:43.289772 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.289751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:09:43.375899 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.375865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a251b7a8-84c4-4d1c-a731-afc7568c32e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.375899 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.375902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-web-config\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.375923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a251b7a8-84c4-4d1c-a731-afc7568c32e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.375948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scfmh\" (UniqueName: \"kubernetes.io/projected/a251b7a8-84c4-4d1c-a731-afc7568c32e7-kube-api-access-scfmh\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376065    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376284 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376284 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a251b7a8-84c4-4d1c-a731-afc7568c32e7-config-out\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376284 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376184    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a251b7a8-84c4-4d1c-a731-afc7568c32e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376284 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.376284 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.376261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a251b7a8-84c4-4d1c-a731-afc7568c32e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.477044 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.477012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a251b7a8-84c4-4d1c-a731-afc7568c32e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.477149 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.477064    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a251b7a8-84c4-4d1c-a731-afc7568c32e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.477297 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.477269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-web-config\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.477417 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.477317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a251b7a8-84c4-4d1c-a731-afc7568c32e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.477593 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.477548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scfmh\" (UniqueName: \"kubernetes.io/projected/a251b7a8-84c4-4d1c-a731-afc7568c32e7-kube-api-access-scfmh\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478109 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a251b7a8-84c4-4d1c-a731-afc7568c32e7-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478211 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478211 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478211 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a251b7a8-84c4-4d1c-a731-afc7568c32e7-config-out\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a251b7a8-84c4-4d1c-a731-afc7568c32e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.478378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.478348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.479365 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.477756    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a251b7a8-84c4-4d1c-a731-afc7568c32e7-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.480284 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.480221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-web-config\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.481386 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.480936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.481386 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.481289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a251b7a8-84c4-4d1c-a731-afc7568c32e7-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.481386 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.481336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a251b7a8-84c4-4d1c-a731-afc7568c32e7-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.483827 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.483803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a251b7a8-84c4-4d1c-a731-afc7568c32e7-config-out\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.483996 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.483974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.484233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.484212    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.484343 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.484314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-config-volume\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.484543 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.484518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.484833 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.484813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a251b7a8-84c4-4d1c-a731-afc7568c32e7-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.486523 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.486502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scfmh\" (UniqueName: \"kubernetes.io/projected/a251b7a8-84c4-4d1c-a731-afc7568c32e7-kube-api-access-scfmh\") pod \"alertmanager-main-0\" (UID: \"a251b7a8-84c4-4d1c-a731-afc7568c32e7\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.596308 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.596278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 10 20:09:43.751217 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.751176    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 10 20:09:43.754056 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:43.754024    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda251b7a8_84c4_4d1c_a731_afc7568c32e7.slice/crio-dc00de61b42a3cffcbcf54a531b5de75c7104f41c00276729e86b41a5fd62771 WatchSource:0}: Error finding container dc00de61b42a3cffcbcf54a531b5de75c7104f41c00276729e86b41a5fd62771: Status 404 returned error can't find the container with id dc00de61b42a3cffcbcf54a531b5de75c7104f41c00276729e86b41a5fd62771
Jul 10 20:09:43.785208 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.785131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerStarted","Data":"dc00de61b42a3cffcbcf54a531b5de75c7104f41c00276729e86b41a5fd62771"}
Jul 10 20:09:43.786754 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.786721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lzt8q" event={"ID":"67a7d692-17e3-4349-a6eb-78a4361a27ce","Type":"ContainerStarted","Data":"29fd4866c9c653601de87adc81cbde53e7e04c4c4b50516faa5a3f856a6b1620"}
Jul 10 20:09:43.786916 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.786838    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:09:43.788317 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.788269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-szwhd" event={"ID":"eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64","Type":"ContainerStarted","Data":"87a09551b29f92d0dd6abf6f681cd5743ee3c80f110d2df75193123c02a8931c"}
Jul 10 20:09:43.793428 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.793402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5v6j2" event={"ID":"e68fbbfc-8e68-49fa-8725-9efb03195902","Type":"ContainerStarted","Data":"0cd80e6ae5e54e6a81648d3b6b6513571aaba2c37aa8c5e6ab2a3841493c86fd"}
Jul 10 20:09:43.802345 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.802279    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lzt8q" podStartSLOduration=66.090005226 podStartE2EDuration="1m10.802262961s" totalImagesPullingTime="4.712257735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:43.801169088 +0000 UTC m=+71.026936016" watchObservedRunningTime="2026-07-10 20:09:43.802262961 +0000 UTC m=+71.028029890"
Jul 10 20:09:43.982317 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.982291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:43.984019 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.983988    2580 patch_prober.go:28] interesting pod/image-registry-7898cfcbf4-62vs7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 20:09:43.984131 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.984039    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7" podUID="76fad17c-35dc-4aa6-b566-cfb59a318a33" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 20:09:43.985101 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:43.985082    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2c4bc253-6422-4be4-83d4-6bb829538a24-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-t2c72\" (UID: \"2c4bc253-6422-4be4-83d4-6bb829538a24\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:44.098834 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:44.098756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72"
Jul 10 20:09:44.226078 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:44.225989    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-szwhd" podStartSLOduration=66.150895425 podStartE2EDuration="1m11.225969455s" totalImagesPullingTime="5.07507403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:43.814950882 +0000 UTC m=+71.040717815" watchObservedRunningTime="2026-07-10 20:09:44.225969455 +0000 UTC m=+71.451736384"
Jul 10 20:09:44.226474 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:44.226459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-t2c72"]
Jul 10 20:09:44.229408 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:44.229384    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4bc253_6422_4be4_83d4_6bb829538a24.slice/crio-9e4e1e392e3adf6f00e2a03884ad34a6aece8394c560d8d525b50048f081a203 WatchSource:0}: Error finding container 9e4e1e392e3adf6f00e2a03884ad34a6aece8394c560d8d525b50048f081a203: Status 404 returned error can't find the container with id 9e4e1e392e3adf6f00e2a03884ad34a6aece8394c560d8d525b50048f081a203
Jul 10 20:09:44.798465 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:44.798426    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72" event={"ID":"2c4bc253-6422-4be4-83d4-6bb829538a24","Type":"ContainerStarted","Data":"9e4e1e392e3adf6f00e2a03884ad34a6aece8394c560d8d525b50048f081a203"}
Jul 10 20:09:44.800157 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:44.800129    2580 generic.go:356] "Generic (PLEG): container finished" podID="e68fbbfc-8e68-49fa-8725-9efb03195902" containerID="2dddd6c5867f6eeedc173d50f159668d7f6d039c5c3a6abe747ec7efd72be8f6" exitCode=0
Jul 10 20:09:44.800274 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:44.800215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5v6j2" event={"ID":"e68fbbfc-8e68-49fa-8725-9efb03195902","Type":"ContainerDied","Data":"2dddd6c5867f6eeedc173d50f159668d7f6d039c5c3a6abe747ec7efd72be8f6"}
Jul 10 20:09:45.692624 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:45.692460    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7898cfcbf4-62vs7"
Jul 10 20:09:45.806160 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:45.806121    2580 generic.go:356] "Generic (PLEG): container finished" podID="a251b7a8-84c4-4d1c-a731-afc7568c32e7" containerID="123889725c20ed422222932bc0e1067f55b1dc5aa175d59fb209623ec1f96f3d" exitCode=0
Jul 10 20:09:45.806591 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:45.806232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerDied","Data":"123889725c20ed422222932bc0e1067f55b1dc5aa175d59fb209623ec1f96f3d"}
Jul 10 20:09:45.808521 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:45.808488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72" event={"ID":"2c4bc253-6422-4be4-83d4-6bb829538a24","Type":"ContainerStarted","Data":"c5c5f8e9dcf580257dc0cf13c75cff02cc3e1d0ce3088db9c61cdd690ac902cd"}
Jul 10 20:09:45.811887 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:45.811846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5v6j2" event={"ID":"e68fbbfc-8e68-49fa-8725-9efb03195902","Type":"ContainerStarted","Data":"bc0f3726b2f38506df3bfe306da28f8ef333b22454a8b32ac42797d4db6a303c"}
Jul 10 20:09:45.811988 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:45.811894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5v6j2" event={"ID":"e68fbbfc-8e68-49fa-8725-9efb03195902","Type":"ContainerStarted","Data":"e12d7a6ccf6d90410e88f981d343c90d79731420989c9242e818626a2dc8b0f1"}
Jul 10 20:09:45.856779 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:45.856723    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5v6j2" podStartSLOduration=2.9014419240000002 podStartE2EDuration="3.856697901s" totalImagesPullingTime="889.930801ms" totalInitContainerRuntime="65.325176ms" isStatefulPod=true podCreationTimestamp="2026-07-10 20:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:45.855925235 +0000 UTC m=+73.081692164" watchObservedRunningTime="2026-07-10 20:09:45.856697901 +0000 UTC m=+73.082464831"
Jul 10 20:09:46.819318 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:46.819279    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72" event={"ID":"2c4bc253-6422-4be4-83d4-6bb829538a24","Type":"ContainerStarted","Data":"b30d4cd2f267cebf698f6efa13dd08c2d10561cdd4635bb1e608b4549c39e89b"}
Jul 10 20:09:46.819785 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:46.819378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72" event={"ID":"2c4bc253-6422-4be4-83d4-6bb829538a24","Type":"ContainerStarted","Data":"a95056969c63a7ce720d1ecaca4eacb586034e2fd6fb749b0733001fa53cb932"}
Jul 10 20:09:46.839062 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:46.839005    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-t2c72" podStartSLOduration=3.4126903840000002 podStartE2EDuration="4.838984125s" totalImagesPullingTime="1.426293741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:46.836922741 +0000 UTC m=+74.062689671" watchObservedRunningTime="2026-07-10 20:09:46.838984125 +0000 UTC m=+74.064751053"
Jul 10 20:09:47.449045 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.449020    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"]
Jul 10 20:09:47.453322 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.453301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.455834 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.455812    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 10 20:09:47.455834 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.455828    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 10 20:09:47.456000 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.455868    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 10 20:09:47.456000 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.455943    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-zmz4s\""
Jul 10 20:09:47.456000 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.455981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 10 20:09:47.456148 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.456109    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 10 20:09:47.460784 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.460766    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 10 20:09:47.467914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.467889    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"]
Jul 10 20:09:47.616414 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-federate-client-tls\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.616585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-serving-certs-ca-bundle\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.616585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616449    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.616585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnkpv\" (UniqueName: \"kubernetes.io/projected/9c35ba24-bb48-420b-9c04-57c89b1f6a80-kube-api-access-bnkpv\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.616585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-secret-telemeter-client\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.616585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.616790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-telemeter-client-tls\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.616790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.616692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-metrics-client-ca\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717399 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-secret-telemeter-client\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717399 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717439    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-telemeter-client-tls\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-metrics-client-ca\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-federate-client-tls\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717819 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-serving-certs-ca-bundle\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717819 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.717819 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.717796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnkpv\" (UniqueName: \"kubernetes.io/projected/9c35ba24-bb48-420b-9c04-57c89b1f6a80-kube-api-access-bnkpv\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.718317 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.718289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-metrics-client-ca\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.718441 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.718369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-serving-certs-ca-bundle\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.718640 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.718615    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c35ba24-bb48-420b-9c04-57c89b1f6a80-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.720123 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.720092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.720123 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.720108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-federate-client-tls\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.720269 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.720092    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-secret-telemeter-client\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.720269 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.720173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c35ba24-bb48-420b-9c04-57c89b1f6a80-telemeter-client-tls\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.725042 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.725020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnkpv\" (UniqueName: \"kubernetes.io/projected/9c35ba24-bb48-420b-9c04-57c89b1f6a80-kube-api-access-bnkpv\") pod \"telemeter-client-85b6ddc97b-2ttrw\" (UID: \"9c35ba24-bb48-420b-9c04-57c89b1f6a80\") " pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.790509 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.790478    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"
Jul 10 20:09:47.826201 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.826168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerStarted","Data":"a211400d101bbdfea086845f9674845e01746dbf0b305d8a2fe43847e41d90a2"}
Jul 10 20:09:47.826572 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.826237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerStarted","Data":"585e8c9e8932e5fca7452852823df6753858fdc9378ae4795ca07dcb56a740ca"}
Jul 10 20:09:47.826572 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.826263    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerStarted","Data":"693684b5daed7b8b45e37da3a68cf3524efa3c6bf9049499ecb33527f6ef9a0a"}
Jul 10 20:09:47.826572 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.826286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerStarted","Data":"968283bc53c43af6bb601f9f872a2ccc5d08ca2cdaf0c23b8a464460e72b0aa1"}
Jul 10 20:09:47.826572 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.826310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerStarted","Data":"c83c2ee2036906c5431899f158b7f79a3c49510df6b1023d0910612d80dd95d2"}
Jul 10 20:09:47.940841 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:47.940811    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw"]
Jul 10 20:09:47.943438 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:09:47.943414    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c35ba24_bb48_420b_9c04_57c89b1f6a80.slice/crio-438d3e40abc040e978e7a33ef05d488efdafa3c324403097633d52ab057f0d5c WatchSource:0}: Error finding container 438d3e40abc040e978e7a33ef05d488efdafa3c324403097633d52ab057f0d5c: Status 404 returned error can't find the container with id 438d3e40abc040e978e7a33ef05d488efdafa3c324403097633d52ab057f0d5c
Jul 10 20:09:48.835227 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:48.835168    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a251b7a8-84c4-4d1c-a731-afc7568c32e7","Type":"ContainerStarted","Data":"b59fdcf2d1f1ccd5e3b1a4c3b9183766e6010c1c189a57137c1ab07f17abcea4"}
Jul 10 20:09:48.836634 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:48.836607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw" event={"ID":"9c35ba24-bb48-420b-9c04-57c89b1f6a80","Type":"ContainerStarted","Data":"438d3e40abc040e978e7a33ef05d488efdafa3c324403097633d52ab057f0d5c"}
Jul 10 20:09:48.862335 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:48.862272    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.770794593 podStartE2EDuration="5.862250666s" totalImagesPullingTime="4.072483614s" totalInitContainerRuntime="18.972459ms" isStatefulPod=false podCreationTimestamp="2026-07-10 20:09:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:48.860514731 +0000 UTC m=+76.086281661" watchObservedRunningTime="2026-07-10 20:09:48.862250666 +0000 UTC m=+76.088017595"
Jul 10 20:09:50.847636 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:50.847604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw" event={"ID":"9c35ba24-bb48-420b-9c04-57c89b1f6a80","Type":"ContainerStarted","Data":"4908906c5ac9bfc61ba93d3192ed8b89e3f814b65aeb66b3e20609f62227a266"}
Jul 10 20:09:50.848060 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:50.847658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw" event={"ID":"9c35ba24-bb48-420b-9c04-57c89b1f6a80","Type":"ContainerStarted","Data":"9b0fa28095ab9640f4ba06edb3fc411612e4577fe418c8c2a35b1ef389d93cb8"}
Jul 10 20:09:50.848060 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:50.847707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw" event={"ID":"9c35ba24-bb48-420b-9c04-57c89b1f6a80","Type":"ContainerStarted","Data":"0472e2ee1cc6a7e46e5bee704ca2c16aa036bfda6249762d3388eea53a972b73"}
Jul 10 20:09:50.870730 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:09:50.870661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85b6ddc97b-2ttrw" podStartSLOduration=1.45825416 podStartE2EDuration="3.870644092s" totalImagesPullingTime="2.412389932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:09:50.868927474 +0000 UTC m=+78.094694404" watchObservedRunningTime="2026-07-10 20:09:50.870644092 +0000 UTC m=+78.096411018"
Jul 10 20:10:14.802981 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:10:14.802953    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lzt8q"
Jul 10 20:10:17.125546 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:10:17.125514    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fl6wl_39306aff-c570-417f-9d22-7db335d5909e/serve-healthcheck-canary/0.log"
Jul 10 20:12:01.710093 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.710057    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"]
Jul 10 20:12:01.713235 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.713216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.715814 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.715789    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 20:12:01.715917 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.715809    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jsrbm\""
Jul 10 20:12:01.715917 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.715838    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 20:12:01.722244 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.722218    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"]
Jul 10 20:12:01.828989 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.828951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.829161 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.829023    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5597f\" (UniqueName: \"kubernetes.io/projected/d9178ce5-d05a-4e46-ad83-20f43ffe1684-kube-api-access-5597f\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.829161 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.829056    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.930018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.929976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5597f\" (UniqueName: \"kubernetes.io/projected/d9178ce5-d05a-4e46-ad83-20f43ffe1684-kube-api-access-5597f\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.930018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.930024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.930274 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.930061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.930488 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.930469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.930560 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.930491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:01.938715 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:01.938663    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5597f\" (UniqueName: \"kubernetes.io/projected/d9178ce5-d05a-4e46-ad83-20f43ffe1684-kube-api-access-5597f\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:02.023735 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:02.023705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:02.151585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:02.151552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"]
Jul 10 20:12:02.156132 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:12:02.156101    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9178ce5_d05a_4e46_ad83_20f43ffe1684.slice/crio-31776535500a951e28f9c891b0765930064ca34282f47c479b646138319eee51 WatchSource:0}: Error finding container 31776535500a951e28f9c891b0765930064ca34282f47c479b646138319eee51: Status 404 returned error can't find the container with id 31776535500a951e28f9c891b0765930064ca34282f47c479b646138319eee51
Jul 10 20:12:02.320468 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:02.320387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j" event={"ID":"d9178ce5-d05a-4e46-ad83-20f43ffe1684","Type":"ContainerStarted","Data":"31776535500a951e28f9c891b0765930064ca34282f47c479b646138319eee51"}
Jul 10 20:12:10.352649 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:10.352614    2580 generic.go:356] "Generic (PLEG): container finished" podID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerID="5ed566c09cd6cb86b25429f5fe0b087dae25a9f6bac98b437d2abf0ed3197dc1" exitCode=0
Jul 10 20:12:10.352649 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:10.352653    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j" event={"ID":"d9178ce5-d05a-4e46-ad83-20f43ffe1684","Type":"ContainerDied","Data":"5ed566c09cd6cb86b25429f5fe0b087dae25a9f6bac98b437d2abf0ed3197dc1"}
Jul 10 20:12:13.364890 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:13.364853    2580 generic.go:356] "Generic (PLEG): container finished" podID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerID="fb8f4076096be1160d83fd13451a601bdd0a922cc3fefc1affd420c83c04b74d" exitCode=0
Jul 10 20:12:13.365281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:13.364927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j" event={"ID":"d9178ce5-d05a-4e46-ad83-20f43ffe1684","Type":"ContainerDied","Data":"fb8f4076096be1160d83fd13451a601bdd0a922cc3fefc1affd420c83c04b74d"}
Jul 10 20:12:21.399454 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:21.399420    2580 generic.go:356] "Generic (PLEG): container finished" podID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerID="2ce50c5d75e9e97ad95893ef21d681e5e3c8b3e5e34982270300256f3077aa18" exitCode=0
Jul 10 20:12:21.399844 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:21.399493    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j" event={"ID":"d9178ce5-d05a-4e46-ad83-20f43ffe1684","Type":"ContainerDied","Data":"2ce50c5d75e9e97ad95893ef21d681e5e3c8b3e5e34982270300256f3077aa18"}
Jul 10 20:12:22.531293 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.531266    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:22.615474 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.615440    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-bundle\") pod \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") "
Jul 10 20:12:22.615638 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.615482    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5597f\" (UniqueName: \"kubernetes.io/projected/d9178ce5-d05a-4e46-ad83-20f43ffe1684-kube-api-access-5597f\") pod \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") "
Jul 10 20:12:22.615638 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.615512    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-util\") pod \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\" (UID: \"d9178ce5-d05a-4e46-ad83-20f43ffe1684\") "
Jul 10 20:12:22.616147 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.616120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-bundle" (OuterVolumeSpecName: "bundle") pod "d9178ce5-d05a-4e46-ad83-20f43ffe1684" (UID: "d9178ce5-d05a-4e46-ad83-20f43ffe1684"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:12:22.617665 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.617623    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9178ce5-d05a-4e46-ad83-20f43ffe1684-kube-api-access-5597f" (OuterVolumeSpecName: "kube-api-access-5597f") pod "d9178ce5-d05a-4e46-ad83-20f43ffe1684" (UID: "d9178ce5-d05a-4e46-ad83-20f43ffe1684"). InnerVolumeSpecName "kube-api-access-5597f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:12:22.619738 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.619711    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-util" (OuterVolumeSpecName: "util") pod "d9178ce5-d05a-4e46-ad83-20f43ffe1684" (UID: "d9178ce5-d05a-4e46-ad83-20f43ffe1684"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:12:22.716847 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.716739    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-bundle\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:22.716847 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.716784    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5597f\" (UniqueName: \"kubernetes.io/projected/d9178ce5-d05a-4e46-ad83-20f43ffe1684-kube-api-access-5597f\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:22.716847 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:22.716798    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9178ce5-d05a-4e46-ad83-20f43ffe1684-util\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:23.409970 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:23.409939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j" event={"ID":"d9178ce5-d05a-4e46-ad83-20f43ffe1684","Type":"ContainerDied","Data":"31776535500a951e28f9c891b0765930064ca34282f47c479b646138319eee51"}
Jul 10 20:12:23.410125 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:23.409984    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31776535500a951e28f9c891b0765930064ca34282f47c479b646138319eee51"
Jul 10 20:12:23.410125 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:23.409986    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540w8j6j"
Jul 10 20:12:29.656217 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.656176    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"]
Jul 10 20:12:29.656856 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.656834    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerName="util"
Jul 10 20:12:29.656961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.656860    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerName="util"
Jul 10 20:12:29.656961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.656874    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerName="pull"
Jul 10 20:12:29.656961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.656881    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerName="pull"
Jul 10 20:12:29.656961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.656919    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerName="extract"
Jul 10 20:12:29.656961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.656928    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerName="extract"
Jul 10 20:12:29.657238 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.657066    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9178ce5-d05a-4e46-ad83-20f43ffe1684" containerName="extract"
Jul 10 20:12:29.661141 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.661117    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:29.663843 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.663825    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:12:29.663938 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.663850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-26bhd\""
Jul 10 20:12:29.664102 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.664087    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 10 20:12:29.673569 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.673548    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"]
Jul 10 20:12:29.773898 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.773857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvqmn\" (UniqueName: \"kubernetes.io/projected/3a842be4-bcc7-4eef-ab6d-b4ae82d2d222-kube-api-access-dvqmn\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbg94\" (UID: \"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:29.773898 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.773899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a842be4-bcc7-4eef-ab6d-b4ae82d2d222-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbg94\" (UID: \"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:29.875182 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.875148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvqmn\" (UniqueName: \"kubernetes.io/projected/3a842be4-bcc7-4eef-ab6d-b4ae82d2d222-kube-api-access-dvqmn\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbg94\" (UID: \"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:29.875182 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.875185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a842be4-bcc7-4eef-ab6d-b4ae82d2d222-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbg94\" (UID: \"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:29.875571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.875554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3a842be4-bcc7-4eef-ab6d-b4ae82d2d222-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbg94\" (UID: \"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:29.887048 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.887020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvqmn\" (UniqueName: \"kubernetes.io/projected/3a842be4-bcc7-4eef-ab6d-b4ae82d2d222-kube-api-access-dvqmn\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbg94\" (UID: \"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:29.970456 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:29.970370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"
Jul 10 20:12:30.104953 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:30.104930    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94"]
Jul 10 20:12:30.107697 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:12:30.107650    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a842be4_bcc7_4eef_ab6d_b4ae82d2d222.slice/crio-89a6da48dfa4dcba618669bfc28b0ae0aa714d0a1b1942c04b6e899c0b947c70 WatchSource:0}: Error finding container 89a6da48dfa4dcba618669bfc28b0ae0aa714d0a1b1942c04b6e899c0b947c70: Status 404 returned error can't find the container with id 89a6da48dfa4dcba618669bfc28b0ae0aa714d0a1b1942c04b6e899c0b947c70
Jul 10 20:12:30.436836 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:30.436806    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94" event={"ID":"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222","Type":"ContainerStarted","Data":"89a6da48dfa4dcba618669bfc28b0ae0aa714d0a1b1942c04b6e899c0b947c70"}
Jul 10 20:12:32.446956 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:32.446922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94" event={"ID":"3a842be4-bcc7-4eef-ab6d-b4ae82d2d222","Type":"ContainerStarted","Data":"0c438646e825d08e882c412efb26e0fbf722c897a19b4aa4d4e20e2036fa9bc6"}
Jul 10 20:12:32.469146 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:32.469091    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbg94" podStartSLOduration=1.824489676 podStartE2EDuration="3.469074225s" totalImagesPullingTime="1.644584549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:12:32.467792031 +0000 UTC m=+239.693558959" watchObservedRunningTime="2026-07-10 20:12:32.469074225 +0000 UTC m=+239.694841154"
Jul 10 20:12:33.357413 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.357378    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"]
Jul 10 20:12:33.360764 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.360748    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.364246 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.364224    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jsrbm\""
Jul 10 20:12:33.364526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.364509    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 20:12:33.365331 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.365318    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 20:12:33.373219 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.373196    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"]
Jul 10 20:12:33.510658 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.510621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.511068 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.510702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrnf7\" (UniqueName: \"kubernetes.io/projected/ebe68b00-8c4f-479c-93a9-164118db5739-kube-api-access-wrnf7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.511068 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.510742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.612067 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.611971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrnf7\" (UniqueName: \"kubernetes.io/projected/ebe68b00-8c4f-479c-93a9-164118db5739-kube-api-access-wrnf7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.612067 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.612019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.612067 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.612066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.612441 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.612422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.612496 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.612462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.622409 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.622387    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrnf7\" (UniqueName: \"kubernetes.io/projected/ebe68b00-8c4f-479c-93a9-164118db5739-kube-api-access-wrnf7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.669248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.669210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:33.804169 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:33.804146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"]
Jul 10 20:12:33.806588 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:12:33.806559    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe68b00_8c4f_479c_93a9_164118db5739.slice/crio-7b8724e0f15b6a238119f18a10ccc1b9140d5071deb593592bee651c7d33e479 WatchSource:0}: Error finding container 7b8724e0f15b6a238119f18a10ccc1b9140d5071deb593592bee651c7d33e479: Status 404 returned error can't find the container with id 7b8724e0f15b6a238119f18a10ccc1b9140d5071deb593592bee651c7d33e479
Jul 10 20:12:34.457120 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:34.457087    2580 generic.go:356] "Generic (PLEG): container finished" podID="ebe68b00-8c4f-479c-93a9-164118db5739" containerID="1740be249152851e3d6458c8d28287ccb96170c6604dd70d0553d450040ccdcc" exitCode=0
Jul 10 20:12:34.457295 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:34.457139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v" event={"ID":"ebe68b00-8c4f-479c-93a9-164118db5739","Type":"ContainerDied","Data":"1740be249152851e3d6458c8d28287ccb96170c6604dd70d0553d450040ccdcc"}
Jul 10 20:12:34.457295 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:34.457175    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v" event={"ID":"ebe68b00-8c4f-479c-93a9-164118db5739","Type":"ContainerStarted","Data":"7b8724e0f15b6a238119f18a10ccc1b9140d5071deb593592bee651c7d33e479"}
Jul 10 20:12:36.041686 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.041639    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r687z"]
Jul 10 20:12:36.053591 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.053560    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.058868 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.058843    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rghvv\""
Jul 10 20:12:36.059018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.058846    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 10 20:12:36.062279 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.062258    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 10 20:12:36.078570 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.078539    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r687z"]
Jul 10 20:12:36.131916 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.131887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrbwc\" (UniqueName: \"kubernetes.io/projected/3271e63b-b199-4c03-a502-57793570a1cf-kube-api-access-nrbwc\") pod \"cert-manager-webhook-6486ffd997-r687z\" (UID: \"3271e63b-b199-4c03-a502-57793570a1cf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.132089 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.131967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3271e63b-b199-4c03-a502-57793570a1cf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r687z\" (UID: \"3271e63b-b199-4c03-a502-57793570a1cf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.232296 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.232275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrbwc\" (UniqueName: \"kubernetes.io/projected/3271e63b-b199-4c03-a502-57793570a1cf-kube-api-access-nrbwc\") pod \"cert-manager-webhook-6486ffd997-r687z\" (UID: \"3271e63b-b199-4c03-a502-57793570a1cf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.232412 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.232319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3271e63b-b199-4c03-a502-57793570a1cf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r687z\" (UID: \"3271e63b-b199-4c03-a502-57793570a1cf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.240880 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.240854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3271e63b-b199-4c03-a502-57793570a1cf-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r687z\" (UID: \"3271e63b-b199-4c03-a502-57793570a1cf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.241062 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.241038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrbwc\" (UniqueName: \"kubernetes.io/projected/3271e63b-b199-4c03-a502-57793570a1cf-kube-api-access-nrbwc\") pod \"cert-manager-webhook-6486ffd997-r687z\" (UID: \"3271e63b-b199-4c03-a502-57793570a1cf\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.372139 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.372102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:36.470357 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.470310    2580 generic.go:356] "Generic (PLEG): container finished" podID="ebe68b00-8c4f-479c-93a9-164118db5739" containerID="7931434e99ce061139152d353bec7e434d36425119129d6b97dd4fcb71b0ea66" exitCode=0
Jul 10 20:12:36.470500 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.470364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v" event={"ID":"ebe68b00-8c4f-479c-93a9-164118db5739","Type":"ContainerDied","Data":"7931434e99ce061139152d353bec7e434d36425119129d6b97dd4fcb71b0ea66"}
Jul 10 20:12:36.501330 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:36.501308    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r687z"]
Jul 10 20:12:36.503885 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:12:36.503859    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3271e63b_b199_4c03_a502_57793570a1cf.slice/crio-0c9a737db464acb4fcb4f2bd14d0e8e19353bb0e262cff165a273430ad4bcdc7 WatchSource:0}: Error finding container 0c9a737db464acb4fcb4f2bd14d0e8e19353bb0e262cff165a273430ad4bcdc7: Status 404 returned error can't find the container with id 0c9a737db464acb4fcb4f2bd14d0e8e19353bb0e262cff165a273430ad4bcdc7
Jul 10 20:12:37.477523 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:37.477491    2580 generic.go:356] "Generic (PLEG): container finished" podID="ebe68b00-8c4f-479c-93a9-164118db5739" containerID="8fef55de6bf5b4e08a594df2724e18e62d9ca0c86ce626da9af2331d4bd6825f" exitCode=0
Jul 10 20:12:37.477988 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:37.477583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v" event={"ID":"ebe68b00-8c4f-479c-93a9-164118db5739","Type":"ContainerDied","Data":"8fef55de6bf5b4e08a594df2724e18e62d9ca0c86ce626da9af2331d4bd6825f"}
Jul 10 20:12:37.478966 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:37.478934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r687z" event={"ID":"3271e63b-b199-4c03-a502-57793570a1cf","Type":"ContainerStarted","Data":"0c9a737db464acb4fcb4f2bd14d0e8e19353bb0e262cff165a273430ad4bcdc7"}
Jul 10 20:12:39.081062 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.081033    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:39.154767 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.154742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrnf7\" (UniqueName: \"kubernetes.io/projected/ebe68b00-8c4f-479c-93a9-164118db5739-kube-api-access-wrnf7\") pod \"ebe68b00-8c4f-479c-93a9-164118db5739\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") "
Jul 10 20:12:39.154880 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.154793    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-util\") pod \"ebe68b00-8c4f-479c-93a9-164118db5739\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") "
Jul 10 20:12:39.154880 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.154833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-bundle\") pod \"ebe68b00-8c4f-479c-93a9-164118db5739\" (UID: \"ebe68b00-8c4f-479c-93a9-164118db5739\") "
Jul 10 20:12:39.155237 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.155207    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-bundle" (OuterVolumeSpecName: "bundle") pod "ebe68b00-8c4f-479c-93a9-164118db5739" (UID: "ebe68b00-8c4f-479c-93a9-164118db5739"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:12:39.156784 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.156754    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe68b00-8c4f-479c-93a9-164118db5739-kube-api-access-wrnf7" (OuterVolumeSpecName: "kube-api-access-wrnf7") pod "ebe68b00-8c4f-479c-93a9-164118db5739" (UID: "ebe68b00-8c4f-479c-93a9-164118db5739"). InnerVolumeSpecName "kube-api-access-wrnf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:12:39.160219 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.160189    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-util" (OuterVolumeSpecName: "util") pod "ebe68b00-8c4f-479c-93a9-164118db5739" (UID: "ebe68b00-8c4f-479c-93a9-164118db5739"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:12:39.255424 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.255387    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrnf7\" (UniqueName: \"kubernetes.io/projected/ebe68b00-8c4f-479c-93a9-164118db5739-kube-api-access-wrnf7\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:39.255424 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.255414    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-util\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:39.255424 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.255423    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebe68b00-8c4f-479c-93a9-164118db5739-bundle\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:39.489248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.489173    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v"
Jul 10 20:12:39.489388 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.489165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvqq2v" event={"ID":"ebe68b00-8c4f-479c-93a9-164118db5739","Type":"ContainerDied","Data":"7b8724e0f15b6a238119f18a10ccc1b9140d5071deb593592bee651c7d33e479"}
Jul 10 20:12:39.489388 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.489296    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b8724e0f15b6a238119f18a10ccc1b9140d5071deb593592bee651c7d33e479"
Jul 10 20:12:39.490612 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.490589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r687z" event={"ID":"3271e63b-b199-4c03-a502-57793570a1cf","Type":"ContainerStarted","Data":"9b334762ef9701d4e7e1cf0bb34b3041fbf73ba7077b22aac1bf3a4cfa462ad6"}
Jul 10 20:12:39.490752 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.490698    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:39.505922 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:39.505875    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-r687z" podStartSLOduration=1.88656793 podStartE2EDuration="4.505858445s" totalImagesPullingTime="2.619290515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:12:39.505028143 +0000 UTC m=+246.730795073" watchObservedRunningTime="2026-07-10 20:12:39.505858445 +0000 UTC m=+246.731625374"
Jul 10 20:12:40.366253 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366218    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"]
Jul 10 20:12:40.366639 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366481    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe68b00-8c4f-479c-93a9-164118db5739" containerName="util"
Jul 10 20:12:40.366639 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366492    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe68b00-8c4f-479c-93a9-164118db5739" containerName="util"
Jul 10 20:12:40.366639 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366501    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe68b00-8c4f-479c-93a9-164118db5739" containerName="extract"
Jul 10 20:12:40.366639 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366507    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe68b00-8c4f-479c-93a9-164118db5739" containerName="extract"
Jul 10 20:12:40.366639 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366526    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe68b00-8c4f-479c-93a9-164118db5739" containerName="pull"
Jul 10 20:12:40.366639 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366531    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe68b00-8c4f-479c-93a9-164118db5739" containerName="pull"
Jul 10 20:12:40.366639 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.366574    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe68b00-8c4f-479c-93a9-164118db5739" containerName="extract"
Jul 10 20:12:40.369536 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.369516    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.372200 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.372179    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dxpcr\""
Jul 10 20:12:40.378095 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.378073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"]
Jul 10 20:12:40.464787 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.464754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfp6w\" (UniqueName: \"kubernetes.io/projected/fbd00ca6-3d72-461e-83b1-09a36e550099-kube-api-access-dfp6w\") pod \"cert-manager-cainjector-559cdd8c87-wq4tn\" (UID: \"fbd00ca6-3d72-461e-83b1-09a36e550099\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.464958 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.464820    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbd00ca6-3d72-461e-83b1-09a36e550099-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wq4tn\" (UID: \"fbd00ca6-3d72-461e-83b1-09a36e550099\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.566076 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.566040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfp6w\" (UniqueName: \"kubernetes.io/projected/fbd00ca6-3d72-461e-83b1-09a36e550099-kube-api-access-dfp6w\") pod \"cert-manager-cainjector-559cdd8c87-wq4tn\" (UID: \"fbd00ca6-3d72-461e-83b1-09a36e550099\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.566250 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.566107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbd00ca6-3d72-461e-83b1-09a36e550099-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wq4tn\" (UID: \"fbd00ca6-3d72-461e-83b1-09a36e550099\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.575059 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.575032    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fbd00ca6-3d72-461e-83b1-09a36e550099-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wq4tn\" (UID: \"fbd00ca6-3d72-461e-83b1-09a36e550099\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.575207 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.575154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfp6w\" (UniqueName: \"kubernetes.io/projected/fbd00ca6-3d72-461e-83b1-09a36e550099-kube-api-access-dfp6w\") pod \"cert-manager-cainjector-559cdd8c87-wq4tn\" (UID: \"fbd00ca6-3d72-461e-83b1-09a36e550099\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.679365 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.679273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"
Jul 10 20:12:40.803353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:40.803329    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn"]
Jul 10 20:12:40.805596 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:12:40.805574    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbd00ca6_3d72_461e_83b1_09a36e550099.slice/crio-3f44be394e52aa0eaa9020cf413d84585ad0020d533198a2a9a1a728e686b4ee WatchSource:0}: Error finding container 3f44be394e52aa0eaa9020cf413d84585ad0020d533198a2a9a1a728e686b4ee: Status 404 returned error can't find the container with id 3f44be394e52aa0eaa9020cf413d84585ad0020d533198a2a9a1a728e686b4ee
Jul 10 20:12:41.504058 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:41.504028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn" event={"ID":"fbd00ca6-3d72-461e-83b1-09a36e550099","Type":"ContainerStarted","Data":"1e5dff39f6ca2cd0a4096c74fa4bed136f55a4a8cf4e1d1a468aeb05debf684b"}
Jul 10 20:12:41.504433 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:41.504071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn" event={"ID":"fbd00ca6-3d72-461e-83b1-09a36e550099","Type":"ContainerStarted","Data":"3f44be394e52aa0eaa9020cf413d84585ad0020d533198a2a9a1a728e686b4ee"}
Jul 10 20:12:41.520924 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:41.520876    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wq4tn" podStartSLOduration=1.520860448 podStartE2EDuration="1.520860448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:12:41.519497354 +0000 UTC m=+248.745264349" watchObservedRunningTime="2026-07-10 20:12:41.520860448 +0000 UTC m=+248.746627378"
Jul 10 20:12:45.497293 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:45.497262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r687z"
Jul 10 20:12:49.997435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:49.997404    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"]
Jul 10 20:12:50.000798 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.000782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.003255 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.003237    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 10 20:12:50.004260 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.004246    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 10 20:12:50.004326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.004258    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jsrbm\""
Jul 10 20:12:50.008926 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.008905    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"]
Jul 10 20:12:50.039101 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.039069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pv5g\" (UniqueName: \"kubernetes.io/projected/0d79d569-f917-4cba-9b99-e0fdaabd699d-kube-api-access-6pv5g\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.039101 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.039105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.039319 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.039143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.140507 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.140471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pv5g\" (UniqueName: \"kubernetes.io/projected/0d79d569-f917-4cba-9b99-e0fdaabd699d-kube-api-access-6pv5g\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.140730 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.140517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.140730 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.140578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.140946 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.140924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.141009 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.140957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.149087 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.149066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pv5g\" (UniqueName: \"kubernetes.io/projected/0d79d569-f917-4cba-9b99-e0fdaabd699d-kube-api-access-6pv5g\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.310756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.310723    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:50.438469 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.438441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"]
Jul 10 20:12:50.440454 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:12:50.440432    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d79d569_f917_4cba_9b99_e0fdaabd699d.slice/crio-600e83af07a9b27342620b79657e59987eac7af0fdfe173b3a4e268116cdf1dc WatchSource:0}: Error finding container 600e83af07a9b27342620b79657e59987eac7af0fdfe173b3a4e268116cdf1dc: Status 404 returned error can't find the container with id 600e83af07a9b27342620b79657e59987eac7af0fdfe173b3a4e268116cdf1dc
Jul 10 20:12:50.546595 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.546553    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj" event={"ID":"0d79d569-f917-4cba-9b99-e0fdaabd699d","Type":"ContainerStarted","Data":"fa4de45d2066db2518382f762fd496ac878dbfe28799e854f73f939f85b830b9"}
Jul 10 20:12:50.546755 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:50.546662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj" event={"ID":"0d79d569-f917-4cba-9b99-e0fdaabd699d","Type":"ContainerStarted","Data":"600e83af07a9b27342620b79657e59987eac7af0fdfe173b3a4e268116cdf1dc"}
Jul 10 20:12:51.554808 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:51.554773    2580 generic.go:356] "Generic (PLEG): container finished" podID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerID="fa4de45d2066db2518382f762fd496ac878dbfe28799e854f73f939f85b830b9" exitCode=0
Jul 10 20:12:51.554808 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:51.554802    2580 generic.go:356] "Generic (PLEG): container finished" podID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerID="bc7474167ea56496e008961ddefc7073a9d721b38b09b865fe666602398208a6" exitCode=0
Jul 10 20:12:51.555204 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:51.554844    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj" event={"ID":"0d79d569-f917-4cba-9b99-e0fdaabd699d","Type":"ContainerDied","Data":"fa4de45d2066db2518382f762fd496ac878dbfe28799e854f73f939f85b830b9"}
Jul 10 20:12:51.555204 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:51.554887    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj" event={"ID":"0d79d569-f917-4cba-9b99-e0fdaabd699d","Type":"ContainerDied","Data":"bc7474167ea56496e008961ddefc7073a9d721b38b09b865fe666602398208a6"}
Jul 10 20:12:52.561492 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:52.561461    2580 generic.go:356] "Generic (PLEG): container finished" podID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerID="f7d3b124c1f01c19bae9f6a077d115757e4fffa3c43429ec7542a3605903786d" exitCode=0
Jul 10 20:12:52.561888 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:52.561515    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj" event={"ID":"0d79d569-f917-4cba-9b99-e0fdaabd699d","Type":"ContainerDied","Data":"f7d3b124c1f01c19bae9f6a077d115757e4fffa3c43429ec7542a3605903786d"}
Jul 10 20:12:53.713468 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.713424    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:53.769471 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.769437    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-util\") pod \"0d79d569-f917-4cba-9b99-e0fdaabd699d\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") "
Jul 10 20:12:53.769610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.769507    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pv5g\" (UniqueName: \"kubernetes.io/projected/0d79d569-f917-4cba-9b99-e0fdaabd699d-kube-api-access-6pv5g\") pod \"0d79d569-f917-4cba-9b99-e0fdaabd699d\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") "
Jul 10 20:12:53.769610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.769533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-bundle\") pod \"0d79d569-f917-4cba-9b99-e0fdaabd699d\" (UID: \"0d79d569-f917-4cba-9b99-e0fdaabd699d\") "
Jul 10 20:12:53.770286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.770258    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-bundle" (OuterVolumeSpecName: "bundle") pod "0d79d569-f917-4cba-9b99-e0fdaabd699d" (UID: "0d79d569-f917-4cba-9b99-e0fdaabd699d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:12:53.771600 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.771573    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d79d569-f917-4cba-9b99-e0fdaabd699d-kube-api-access-6pv5g" (OuterVolumeSpecName: "kube-api-access-6pv5g") pod "0d79d569-f917-4cba-9b99-e0fdaabd699d" (UID: "0d79d569-f917-4cba-9b99-e0fdaabd699d"). InnerVolumeSpecName "kube-api-access-6pv5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:12:53.774948 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.774912    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-util" (OuterVolumeSpecName: "util") pod "0d79d569-f917-4cba-9b99-e0fdaabd699d" (UID: "0d79d569-f917-4cba-9b99-e0fdaabd699d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:12:53.870216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.870126    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-util\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:53.870216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.870158    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pv5g\" (UniqueName: \"kubernetes.io/projected/0d79d569-f917-4cba-9b99-e0fdaabd699d-kube-api-access-6pv5g\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:53.870216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:53.870167    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d79d569-f917-4cba-9b99-e0fdaabd699d-bundle\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:12:54.052885 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.052853    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbkbj"]
Jul 10 20:12:54.053120 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.053108    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerName="extract"
Jul 10 20:12:54.053163 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.053123    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerName="extract"
Jul 10 20:12:54.053163 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.053134    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerName="pull"
Jul 10 20:12:54.053163 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.053140    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerName="pull"
Jul 10 20:12:54.053163 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.053153    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerName="util"
Jul 10 20:12:54.053163 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.053157    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerName="util"
Jul 10 20:12:54.053306 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.053201    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d79d569-f917-4cba-9b99-e0fdaabd699d" containerName="extract"
Jul 10 20:12:54.074140 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.074112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbkbj"]
Jul 10 20:12:54.074280 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.074222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.077466 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.077224    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-s76xv\""
Jul 10 20:12:54.172767 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.172652    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlmgz\" (UniqueName: \"kubernetes.io/projected/07420827-fc7e-45fb-97b5-74a37b492adb-kube-api-access-mlmgz\") pod \"cert-manager-55b4d954d5-xbkbj\" (UID: \"07420827-fc7e-45fb-97b5-74a37b492adb\") " pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.172914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.172789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07420827-fc7e-45fb-97b5-74a37b492adb-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbkbj\" (UID: \"07420827-fc7e-45fb-97b5-74a37b492adb\") " pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.273818 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.273783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07420827-fc7e-45fb-97b5-74a37b492adb-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbkbj\" (UID: \"07420827-fc7e-45fb-97b5-74a37b492adb\") " pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.273974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.273848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlmgz\" (UniqueName: \"kubernetes.io/projected/07420827-fc7e-45fb-97b5-74a37b492adb-kube-api-access-mlmgz\") pod \"cert-manager-55b4d954d5-xbkbj\" (UID: \"07420827-fc7e-45fb-97b5-74a37b492adb\") " pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.281885 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.281851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/07420827-fc7e-45fb-97b5-74a37b492adb-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbkbj\" (UID: \"07420827-fc7e-45fb-97b5-74a37b492adb\") " pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.281994 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.281910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlmgz\" (UniqueName: \"kubernetes.io/projected/07420827-fc7e-45fb-97b5-74a37b492adb-kube-api-access-mlmgz\") pod \"cert-manager-55b4d954d5-xbkbj\" (UID: \"07420827-fc7e-45fb-97b5-74a37b492adb\") " pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.385089 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.385004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xbkbj"
Jul 10 20:12:54.544771 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.544735    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbkbj"]
Jul 10 20:12:54.548001 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:12:54.547973    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07420827_fc7e_45fb_97b5_74a37b492adb.slice/crio-e90f1220a2fae23e68f91a53d865465373875d11327cb75f3f7b0141b1372548 WatchSource:0}: Error finding container e90f1220a2fae23e68f91a53d865465373875d11327cb75f3f7b0141b1372548: Status 404 returned error can't find the container with id e90f1220a2fae23e68f91a53d865465373875d11327cb75f3f7b0141b1372548
Jul 10 20:12:54.573382 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.573360    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj"
Jul 10 20:12:54.573494 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.573359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjkcqj" event={"ID":"0d79d569-f917-4cba-9b99-e0fdaabd699d","Type":"ContainerDied","Data":"600e83af07a9b27342620b79657e59987eac7af0fdfe173b3a4e268116cdf1dc"}
Jul 10 20:12:54.573494 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.573483    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="600e83af07a9b27342620b79657e59987eac7af0fdfe173b3a4e268116cdf1dc"
Jul 10 20:12:54.574606 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:54.574588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xbkbj" event={"ID":"07420827-fc7e-45fb-97b5-74a37b492adb","Type":"ContainerStarted","Data":"e90f1220a2fae23e68f91a53d865465373875d11327cb75f3f7b0141b1372548"}
Jul 10 20:12:55.580608 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:55.580572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xbkbj" event={"ID":"07420827-fc7e-45fb-97b5-74a37b492adb","Type":"ContainerStarted","Data":"bb5cc4d3818063e8b12e6e08f489a1b6343ae01d96d387a8f996a59d2399826e"}
Jul 10 20:12:55.595681 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:12:55.595628    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xbkbj" podStartSLOduration=1.595610879 podStartE2EDuration="1.595610879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:12:55.595228345 +0000 UTC m=+262.820995255" watchObservedRunningTime="2026-07-10 20:12:55.595610879 +0000 UTC m=+262.821377807"
Jul 10 20:13:06.496745 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.496705    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"]
Jul 10 20:13:06.500302 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.500285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.506336 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.506306    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 20:13:06.506504 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.506371    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 20:13:06.506504 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.506487    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qrfp4\""
Jul 10 20:13:06.506636 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.506601    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 20:13:06.506716 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.506683    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 20:13:06.515494 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.515455    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"]
Jul 10 20:13:06.584614 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.584579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrpbb\" (UniqueName: \"kubernetes.io/projected/173bfd55-e541-4e2c-871b-3beb2747e88e-kube-api-access-hrpbb\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.584791 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.584619    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/173bfd55-e541-4e2c-871b-3beb2747e88e-tmp\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.584791 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.584643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/173bfd55-e541-4e2c-871b-3beb2747e88e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.584791 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.584704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/173bfd55-e541-4e2c-871b-3beb2747e88e-manifests\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.584791 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.584752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/173bfd55-e541-4e2c-871b-3beb2747e88e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.685809 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.685770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrpbb\" (UniqueName: \"kubernetes.io/projected/173bfd55-e541-4e2c-871b-3beb2747e88e-kube-api-access-hrpbb\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.685809 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.685812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/173bfd55-e541-4e2c-871b-3beb2747e88e-tmp\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.686051 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.685831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/173bfd55-e541-4e2c-871b-3beb2747e88e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.686051 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.685861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/173bfd55-e541-4e2c-871b-3beb2747e88e-manifests\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.686051 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.685922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/173bfd55-e541-4e2c-871b-3beb2747e88e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.686336 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.686312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/173bfd55-e541-4e2c-871b-3beb2747e88e-manifests\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.688248 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.688226    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/173bfd55-e541-4e2c-871b-3beb2747e88e-tmp\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.688354 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.688326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/173bfd55-e541-4e2c-871b-3beb2747e88e-webhook-cert\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.688412 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.688372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/173bfd55-e541-4e2c-871b-3beb2747e88e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.693877 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.693857    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrpbb\" (UniqueName: \"kubernetes.io/projected/173bfd55-e541-4e2c-871b-3beb2747e88e-kube-api-access-hrpbb\") pod \"opendatahub-operator-controller-manager-5767ffdff4-qkqf6\" (UID: \"173bfd55-e541-4e2c-871b-3beb2747e88e\") " pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.820861 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.820824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:06.962293 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:06.962262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"]
Jul 10 20:13:06.965148 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:13:06.965122    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173bfd55_e541_4e2c_871b_3beb2747e88e.slice/crio-2201920a5a7b7fb7451f91173476c9bfe3113bf63c92c64d3e692a099efe29e9 WatchSource:0}: Error finding container 2201920a5a7b7fb7451f91173476c9bfe3113bf63c92c64d3e692a099efe29e9: Status 404 returned error can't find the container with id 2201920a5a7b7fb7451f91173476c9bfe3113bf63c92c64d3e692a099efe29e9
Jul 10 20:13:07.634040 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:07.634004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6" event={"ID":"173bfd55-e541-4e2c-871b-3beb2747e88e","Type":"ContainerStarted","Data":"2201920a5a7b7fb7451f91173476c9bfe3113bf63c92c64d3e692a099efe29e9"}
Jul 10 20:13:10.653430 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:10.653340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6" event={"ID":"173bfd55-e541-4e2c-871b-3beb2747e88e","Type":"ContainerStarted","Data":"4fef4d9e87cbfea53bf484fa81300a2896f563275a99caf510ed6f083a75e3e9"}
Jul 10 20:13:11.659250 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:11.659215    2580 generic.go:356] "Generic (PLEG): container finished" podID="173bfd55-e541-4e2c-871b-3beb2747e88e" containerID="4fef4d9e87cbfea53bf484fa81300a2896f563275a99caf510ed6f083a75e3e9" exitCode=0
Jul 10 20:13:11.659616 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:11.659295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6" event={"ID":"173bfd55-e541-4e2c-871b-3beb2747e88e","Type":"ContainerDied","Data":"4fef4d9e87cbfea53bf484fa81300a2896f563275a99caf510ed6f083a75e3e9"}
Jul 10 20:13:12.667461 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:12.667420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6" event={"ID":"173bfd55-e541-4e2c-871b-3beb2747e88e","Type":"ContainerStarted","Data":"76c55e89adef612a11de6141900cdf440066ae1185c144f11a7d8c7d44c196d3"}
Jul 10 20:13:12.667903 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:12.667563    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:12.692854 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:12.692802    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6" podStartSLOduration=1.739895061 podStartE2EDuration="6.692784362s" totalImagesPullingTime="3.923299374s" totalInitContainerRuntime="1.029589927s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:13:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:13:12.690535828 +0000 UTC m=+279.916302770" watchObservedRunningTime="2026-07-10 20:13:12.692784362 +0000 UTC m=+279.918551355"
Jul 10 20:13:23.674141 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:23.674102    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5767ffdff4-qkqf6"
Jul 10 20:13:25.457681 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.457633    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"]
Jul 10 20:13:25.461851 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.461826    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.469139 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.469115    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 10 20:13:25.469519 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.469501    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 10 20:13:25.470010 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.469656    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 20:13:25.470164 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.469743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lh8n7\""
Jul 10 20:13:25.470309 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.469846    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 20:13:25.470370 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.469894    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 10 20:13:25.481536 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.481500    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"]
Jul 10 20:13:25.549212 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.549177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-metrics-cert\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.549378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.549235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-manager-config\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.549378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.549260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-cert\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.549378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.549293    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cjcp\" (UniqueName: \"kubernetes.io/projected/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-kube-api-access-7cjcp\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.650559 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.650523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-metrics-cert\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.650784 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.650579    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-manager-config\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.650784 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.650600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-cert\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.650784 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.650726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cjcp\" (UniqueName: \"kubernetes.io/projected/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-kube-api-access-7cjcp\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.651358 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.651336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-manager-config\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.653025 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.653001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-cert\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.653095 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.653034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-metrics-cert\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.682428 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.682392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cjcp\" (UniqueName: \"kubernetes.io/projected/b90491f6-a35b-4bba-8eea-c1eaefcb8c38-kube-api-access-7cjcp\") pod \"lws-controller-manager-7bd7b6bf4b-xf2jm\" (UID: \"b90491f6-a35b-4bba-8eea-c1eaefcb8c38\") " pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.773073 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.773041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:25.944201 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:25.944108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"]
Jul 10 20:13:25.946937 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:13:25.946909    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb90491f6_a35b_4bba_8eea_c1eaefcb8c38.slice/crio-17ff605264a217c94dd382465edbffe2d2f3fe4b4c79057064c0944137cea5c6 WatchSource:0}: Error finding container 17ff605264a217c94dd382465edbffe2d2f3fe4b4c79057064c0944137cea5c6: Status 404 returned error can't find the container with id 17ff605264a217c94dd382465edbffe2d2f3fe4b4c79057064c0944137cea5c6
Jul 10 20:13:26.732445 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:26.732412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm" event={"ID":"b90491f6-a35b-4bba-8eea-c1eaefcb8c38","Type":"ContainerStarted","Data":"17ff605264a217c94dd382465edbffe2d2f3fe4b4c79057064c0944137cea5c6"}
Jul 10 20:13:28.743746 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:28.743709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm" event={"ID":"b90491f6-a35b-4bba-8eea-c1eaefcb8c38","Type":"ContainerStarted","Data":"eb6aba3ad466ac7f2bbcfdd66b65580e23f2d7de4e89ceb8c4752c8981867fff"}
Jul 10 20:13:28.744234 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:28.743791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:28.761357 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:28.761306    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm" podStartSLOduration=1.962555249 podStartE2EDuration="3.761289529s" totalImagesPullingTime="1.79873428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:13:28.760084669 +0000 UTC m=+295.985851602" watchObservedRunningTime="2026-07-10 20:13:28.761289529 +0000 UTC m=+295.987056749"
Jul 10 20:13:33.134860 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:33.134685    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:13:33.138937 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:33.138913    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 20:13:33.141533 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:33.141516    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:13:38.580102 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.580064    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"]
Jul 10 20:13:38.585931 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.585906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.588491 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.588471    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 10 20:13:38.588785 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.588767    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 10 20:13:38.589236 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.589191    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5dt84\""
Jul 10 20:13:38.589327 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.589231    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 20:13:38.589480 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.589465    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 20:13:38.589552 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.589466    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 10 20:13:38.589621 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.589603    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 20:13:38.595660 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.595641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"]
Jul 10 20:13:38.661904 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.661863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.661904 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.661909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h28k\" (UniqueName: \"kubernetes.io/projected/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-kube-api-access-6h28k\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.662123 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.661950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.662123 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.662061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.662200 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.662120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.662200 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.662180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.662272 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.662227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.762625 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.762588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.762625 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.762630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.762914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.762665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.762914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.762722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h28k\" (UniqueName: \"kubernetes.io/projected/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-kube-api-access-6h28k\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.762914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.762756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.762914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.762794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.762914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.762829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.763389 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.763366    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.765079 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.765052    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.765185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.765053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.765549 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.765526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.765618 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.765532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.771382 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.771362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h28k\" (UniqueName: \"kubernetes.io/projected/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-kube-api-access-6h28k\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.771606 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.771590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/624e411b-2dbe-4a6e-8797-9d0c4ad44a35-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-l7gcj\" (UID: \"624e411b-2dbe-4a6e-8797-9d0c4ad44a35\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:38.895815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:38.895712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:39.030476 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:39.030452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"]
Jul 10 20:13:39.032417 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:13:39.032389    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624e411b_2dbe_4a6e_8797_9d0c4ad44a35.slice/crio-9107bc3cb7ef1972e4a34b5f27cac49496f80d506723f6cca1abe4c62d4cd2e7 WatchSource:0}: Error finding container 9107bc3cb7ef1972e4a34b5f27cac49496f80d506723f6cca1abe4c62d4cd2e7: Status 404 returned error can't find the container with id 9107bc3cb7ef1972e4a34b5f27cac49496f80d506723f6cca1abe4c62d4cd2e7
Jul 10 20:13:39.034313 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:39.034297    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:13:39.751088 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:39.751053    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bd7b6bf4b-xf2jm"
Jul 10 20:13:39.807436 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:39.807373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj" event={"ID":"624e411b-2dbe-4a6e-8797-9d0c4ad44a35","Type":"ContainerStarted","Data":"9107bc3cb7ef1972e4a34b5f27cac49496f80d506723f6cca1abe4c62d4cd2e7"}
Jul 10 20:13:41.782376 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:41.782328    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 20:13:41.782649 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:41.782410    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 20:13:42.822775 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:42.822741    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj" event={"ID":"624e411b-2dbe-4a6e-8797-9d0c4ad44a35","Type":"ContainerStarted","Data":"e584283cef8beeed33ee5b2a18a52e52ea0c67d70b9ccc198b70cd0d806572a3"}
Jul 10 20:13:42.823222 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:42.822888    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:13:42.824547 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:42.824518    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-l7gcj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 20:13:42.824654 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:42.824565    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj" podUID="624e411b-2dbe-4a6e-8797-9d0c4ad44a35" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 20:13:42.845089 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:42.845045    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj" podStartSLOduration=2.097400076 podStartE2EDuration="4.845030605s" totalImagesPullingTime="2.747630529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:13:42.844127494 +0000 UTC m=+310.069894436" watchObservedRunningTime="2026-07-10 20:13:42.845030605 +0000 UTC m=+310.070797533"
Jul 10 20:13:43.828281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:13:43.828252    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-l7gcj"
Jul 10 20:14:42.955014 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.954982    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"]
Jul 10 20:14:42.958372 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.958357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:42.960887 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.960869    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 20:14:42.961824 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.961805    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 20:14:42.961928 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.961806    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7rtwr\""
Jul 10 20:14:42.966012 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.965993    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"]
Jul 10 20:14:42.992896 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.992865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:42.993041 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.992901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:42.993041 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:42.992933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncwwp\" (UniqueName: \"kubernetes.io/projected/abf734a4-c170-4437-8a61-a23790b66e00-kube-api-access-ncwwp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.093826 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.093791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.094019 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.093835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.094019 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.093888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncwwp\" (UniqueName: \"kubernetes.io/projected/abf734a4-c170-4437-8a61-a23790b66e00-kube-api-access-ncwwp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.094180 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.094161    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.094245 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.094190    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.101815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.101793    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncwwp\" (UniqueName: \"kubernetes.io/projected/abf734a4-c170-4437-8a61-a23790b66e00-kube-api-access-ncwwp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.268023 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.268002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:43.402990 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.402933    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"]
Jul 10 20:14:43.405287 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:14:43.405257    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf734a4_c170_4437_8a61_a23790b66e00.slice/crio-72caa6de95a1c863534741c11fa3d1192469d32f48781a24b29193c0ce5035af WatchSource:0}: Error finding container 72caa6de95a1c863534741c11fa3d1192469d32f48781a24b29193c0ce5035af: Status 404 returned error can't find the container with id 72caa6de95a1c863534741c11fa3d1192469d32f48781a24b29193c0ce5035af
Jul 10 20:14:43.754133 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.754098    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"]
Jul 10 20:14:43.757390 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.757374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.764699 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.764563    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"]
Jul 10 20:14:43.800397 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.800285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.800397 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.800325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.800397 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.800342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkfrz\" (UniqueName: \"kubernetes.io/projected/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-kube-api-access-nkfrz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.900838 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.900795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.900838 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.900839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.901036 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.900860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkfrz\" (UniqueName: \"kubernetes.io/projected/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-kube-api-access-nkfrz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.901218 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.901198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.901254 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.901228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:43.908948 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:43.908927    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkfrz\" (UniqueName: \"kubernetes.io/projected/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-kube-api-access-nkfrz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:44.067600 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.067514    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:44.097210 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.097175    2580 generic.go:356] "Generic (PLEG): container finished" podID="abf734a4-c170-4437-8a61-a23790b66e00" containerID="c9ec490b5f22f371dd154f84af87527ddac29b27d5785cc0f6a0037cc9f9a336" exitCode=0
Jul 10 20:14:44.097362 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.097257    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5" event={"ID":"abf734a4-c170-4437-8a61-a23790b66e00","Type":"ContainerDied","Data":"c9ec490b5f22f371dd154f84af87527ddac29b27d5785cc0f6a0037cc9f9a336"}
Jul 10 20:14:44.097362 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.097320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5" event={"ID":"abf734a4-c170-4437-8a61-a23790b66e00","Type":"ContainerStarted","Data":"72caa6de95a1c863534741c11fa3d1192469d32f48781a24b29193c0ce5035af"}
Jul 10 20:14:44.204280 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.204241    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"]
Jul 10 20:14:44.207737 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:14:44.207710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b99e262_2fe2_46ac_ad69_7b8674fad8b5.slice/crio-49e678bd9e776ab358e1e58e2ba0258381f44a58e17666fd32444b175e1f98b9 WatchSource:0}: Error finding container 49e678bd9e776ab358e1e58e2ba0258381f44a58e17666fd32444b175e1f98b9: Status 404 returned error can't find the container with id 49e678bd9e776ab358e1e58e2ba0258381f44a58e17666fd32444b175e1f98b9
Jul 10 20:14:44.366833 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.366751    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"]
Jul 10 20:14:44.370074 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.370055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.377824 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.377798    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"]
Jul 10 20:14:44.405489 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.405460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.405704 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.405502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.405704 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.405612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bnsc\" (UniqueName: \"kubernetes.io/projected/355e96c3-2a65-4e69-9818-c73870010ea0-kube-api-access-9bnsc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.506998 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.506958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.507160 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.507014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bnsc\" (UniqueName: \"kubernetes.io/projected/355e96c3-2a65-4e69-9818-c73870010ea0-kube-api-access-9bnsc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.507160 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.507104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.507417 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.507390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.507458 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.507431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.514577 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.514549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bnsc\" (UniqueName: \"kubernetes.io/projected/355e96c3-2a65-4e69-9818-c73870010ea0-kube-api-access-9bnsc\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.679979 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.679890    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:44.763015 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.762985    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"]
Jul 10 20:14:44.766859 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.766838    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.775700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.775657    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"]
Jul 10 20:14:44.810137 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.810105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ft7g\" (UniqueName: \"kubernetes.io/projected/78375b72-580e-4f47-a6f4-226c88ee9629-kube-api-access-2ft7g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.810313 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.810171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.810313 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.810246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.815546 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.815524    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"]
Jul 10 20:14:44.817461 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:14:44.817435    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355e96c3_2a65_4e69_9818_c73870010ea0.slice/crio-a915f3ea638daebdf3c877a9de5e0036acf3c703052f188fbf0de99a0b852421 WatchSource:0}: Error finding container a915f3ea638daebdf3c877a9de5e0036acf3c703052f188fbf0de99a0b852421: Status 404 returned error can't find the container with id a915f3ea638daebdf3c877a9de5e0036acf3c703052f188fbf0de99a0b852421
Jul 10 20:14:44.911565 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.911540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.911689 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.911626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ft7g\" (UniqueName: \"kubernetes.io/projected/78375b72-580e-4f47-a6f4-226c88ee9629-kube-api-access-2ft7g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.911776 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.911712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.912004 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.911984    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.912113 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.912096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:44.919370 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:44.919350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ft7g\" (UniqueName: \"kubernetes.io/projected/78375b72-580e-4f47-a6f4-226c88ee9629-kube-api-access-2ft7g\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:45.078632 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.078593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:45.103748 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.103717    2580 generic.go:356] "Generic (PLEG): container finished" podID="355e96c3-2a65-4e69-9818-c73870010ea0" containerID="835086faf4c807c33560a0f4990ddbac3a86825ee7dcd163cc611e96d9089a4f" exitCode=0
Jul 10 20:14:45.103901 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.103815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls" event={"ID":"355e96c3-2a65-4e69-9818-c73870010ea0","Type":"ContainerDied","Data":"835086faf4c807c33560a0f4990ddbac3a86825ee7dcd163cc611e96d9089a4f"}
Jul 10 20:14:45.103901 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.103882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls" event={"ID":"355e96c3-2a65-4e69-9818-c73870010ea0","Type":"ContainerStarted","Data":"a915f3ea638daebdf3c877a9de5e0036acf3c703052f188fbf0de99a0b852421"}
Jul 10 20:14:45.105518 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.105493    2580 generic.go:356] "Generic (PLEG): container finished" podID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerID="3e252d115f81ae41fc71ef7a5af3f01675f011e96a54ffbcd06f5e67f68c6d99" exitCode=0
Jul 10 20:14:45.105792 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.105573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll" event={"ID":"9b99e262-2fe2-46ac-ad69-7b8674fad8b5","Type":"ContainerDied","Data":"3e252d115f81ae41fc71ef7a5af3f01675f011e96a54ffbcd06f5e67f68c6d99"}
Jul 10 20:14:45.105792 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.105616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll" event={"ID":"9b99e262-2fe2-46ac-ad69-7b8674fad8b5","Type":"ContainerStarted","Data":"49e678bd9e776ab358e1e58e2ba0258381f44a58e17666fd32444b175e1f98b9"}
Jul 10 20:14:45.107843 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.107819    2580 generic.go:356] "Generic (PLEG): container finished" podID="abf734a4-c170-4437-8a61-a23790b66e00" containerID="8b8052c1652aec08b5ad032b7790fa780c05686d69b2808490e4cd8b155325e2" exitCode=0
Jul 10 20:14:45.107957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.107943    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5" event={"ID":"abf734a4-c170-4437-8a61-a23790b66e00","Type":"ContainerDied","Data":"8b8052c1652aec08b5ad032b7790fa780c05686d69b2808490e4cd8b155325e2"}
Jul 10 20:14:45.217127 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:45.217055    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"]
Jul 10 20:14:45.220053 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:14:45.220024    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78375b72_580e_4f47_a6f4_226c88ee9629.slice/crio-bd6decca26ccee167b19366c9835d7d2d2c7b85f7388685f532affcf04ae6b0b WatchSource:0}: Error finding container bd6decca26ccee167b19366c9835d7d2d2c7b85f7388685f532affcf04ae6b0b: Status 404 returned error can't find the container with id bd6decca26ccee167b19366c9835d7d2d2c7b85f7388685f532affcf04ae6b0b
Jul 10 20:14:46.115954 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:46.115920    2580 generic.go:356] "Generic (PLEG): container finished" podID="355e96c3-2a65-4e69-9818-c73870010ea0" containerID="41f29ad387e20d38e09d80ee9c5420970ffcb0f4eccabed5718cedc1ca0aa179" exitCode=0
Jul 10 20:14:46.116387 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:46.115961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls" event={"ID":"355e96c3-2a65-4e69-9818-c73870010ea0","Type":"ContainerDied","Data":"41f29ad387e20d38e09d80ee9c5420970ffcb0f4eccabed5718cedc1ca0aa179"}
Jul 10 20:14:46.117532 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:46.117507    2580 generic.go:356] "Generic (PLEG): container finished" podID="78375b72-580e-4f47-a6f4-226c88ee9629" containerID="bdbd7e1f18f66dc6045e1fe1363e504eccdb165f8a64de301aa7c4c8ea475121" exitCode=0
Jul 10 20:14:46.117625 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:46.117581    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc" event={"ID":"78375b72-580e-4f47-a6f4-226c88ee9629","Type":"ContainerDied","Data":"bdbd7e1f18f66dc6045e1fe1363e504eccdb165f8a64de301aa7c4c8ea475121"}
Jul 10 20:14:46.117698 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:46.117625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc" event={"ID":"78375b72-580e-4f47-a6f4-226c88ee9629","Type":"ContainerStarted","Data":"bd6decca26ccee167b19366c9835d7d2d2c7b85f7388685f532affcf04ae6b0b"}
Jul 10 20:14:46.120024 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:46.119976    2580 generic.go:356] "Generic (PLEG): container finished" podID="abf734a4-c170-4437-8a61-a23790b66e00" containerID="60f3500c46d14cd995c7f4aa40da3331030e367230161d002f1dfd471ad0b4d8" exitCode=0
Jul 10 20:14:46.120109 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:46.120019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5" event={"ID":"abf734a4-c170-4437-8a61-a23790b66e00","Type":"ContainerDied","Data":"60f3500c46d14cd995c7f4aa40da3331030e367230161d002f1dfd471ad0b4d8"}
Jul 10 20:14:47.126656 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.126565    2580 generic.go:356] "Generic (PLEG): container finished" podID="355e96c3-2a65-4e69-9818-c73870010ea0" containerID="2049fa15b0daae14f7bec5221ccfe3b8db3c869bb44ac910f1c53757181b2e4a" exitCode=0
Jul 10 20:14:47.127072 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.126651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls" event={"ID":"355e96c3-2a65-4e69-9818-c73870010ea0","Type":"ContainerDied","Data":"2049fa15b0daae14f7bec5221ccfe3b8db3c869bb44ac910f1c53757181b2e4a"}
Jul 10 20:14:47.128476 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.128453    2580 generic.go:356] "Generic (PLEG): container finished" podID="78375b72-580e-4f47-a6f4-226c88ee9629" containerID="94a10a7ac33b41effa036a06972e3928835f6e85a18f9a1aa1ea7259828b44f6" exitCode=0
Jul 10 20:14:47.128595 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.128538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc" event={"ID":"78375b72-580e-4f47-a6f4-226c88ee9629","Type":"ContainerDied","Data":"94a10a7ac33b41effa036a06972e3928835f6e85a18f9a1aa1ea7259828b44f6"}
Jul 10 20:14:47.130696 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.130660    2580 generic.go:356] "Generic (PLEG): container finished" podID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerID="c321739b0d3029f0621b2a8eb54163a45eda83062a315188693723c0a6588ea9" exitCode=0
Jul 10 20:14:47.130803 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.130705    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll" event={"ID":"9b99e262-2fe2-46ac-ad69-7b8674fad8b5","Type":"ContainerDied","Data":"c321739b0d3029f0621b2a8eb54163a45eda83062a315188693723c0a6588ea9"}
Jul 10 20:14:47.275769 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.275648    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:47.330947 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.330920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncwwp\" (UniqueName: \"kubernetes.io/projected/abf734a4-c170-4437-8a61-a23790b66e00-kube-api-access-ncwwp\") pod \"abf734a4-c170-4437-8a61-a23790b66e00\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") "
Jul 10 20:14:47.331079 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.330984    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-bundle\") pod \"abf734a4-c170-4437-8a61-a23790b66e00\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") "
Jul 10 20:14:47.331079 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.331047    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-util\") pod \"abf734a4-c170-4437-8a61-a23790b66e00\" (UID: \"abf734a4-c170-4437-8a61-a23790b66e00\") "
Jul 10 20:14:47.331749 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.331720    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-bundle" (OuterVolumeSpecName: "bundle") pod "abf734a4-c170-4437-8a61-a23790b66e00" (UID: "abf734a4-c170-4437-8a61-a23790b66e00"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:47.333200 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.333179    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf734a4-c170-4437-8a61-a23790b66e00-kube-api-access-ncwwp" (OuterVolumeSpecName: "kube-api-access-ncwwp") pod "abf734a4-c170-4437-8a61-a23790b66e00" (UID: "abf734a4-c170-4437-8a61-a23790b66e00"). InnerVolumeSpecName "kube-api-access-ncwwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:14:47.337450 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.337428    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-util" (OuterVolumeSpecName: "util") pod "abf734a4-c170-4437-8a61-a23790b66e00" (UID: "abf734a4-c170-4437-8a61-a23790b66e00"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:47.432303 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.432218    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncwwp\" (UniqueName: \"kubernetes.io/projected/abf734a4-c170-4437-8a61-a23790b66e00-kube-api-access-ncwwp\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:47.432303 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.432250    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-bundle\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:47.432303 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:47.432262    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abf734a4-c170-4437-8a61-a23790b66e00-util\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:48.137651 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.137618    2580 generic.go:356] "Generic (PLEG): container finished" podID="78375b72-580e-4f47-a6f4-226c88ee9629" containerID="a7b7f7173502f446c35b3c3bb1c98646847e6b173361883dbc14bac53fb7ae88" exitCode=0
Jul 10 20:14:48.138095 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.137706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc" event={"ID":"78375b72-580e-4f47-a6f4-226c88ee9629","Type":"ContainerDied","Data":"a7b7f7173502f446c35b3c3bb1c98646847e6b173361883dbc14bac53fb7ae88"}
Jul 10 20:14:48.139693 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.139653    2580 generic.go:356] "Generic (PLEG): container finished" podID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerID="13226e71823135a460e9455650ac5832c936cd3104e76545976a70e76d4cb594" exitCode=0
Jul 10 20:14:48.139795 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.139695    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll" event={"ID":"9b99e262-2fe2-46ac-ad69-7b8674fad8b5","Type":"ContainerDied","Data":"13226e71823135a460e9455650ac5832c936cd3104e76545976a70e76d4cb594"}
Jul 10 20:14:48.141494 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.141477    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5"
Jul 10 20:14:48.141581 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.141506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5" event={"ID":"abf734a4-c170-4437-8a61-a23790b66e00","Type":"ContainerDied","Data":"72caa6de95a1c863534741c11fa3d1192469d32f48781a24b29193c0ce5035af"}
Jul 10 20:14:48.141581 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.141539    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72caa6de95a1c863534741c11fa3d1192469d32f48781a24b29193c0ce5035af"
Jul 10 20:14:48.280026 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.280004    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:48.340456 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.340424    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-bundle\") pod \"355e96c3-2a65-4e69-9818-c73870010ea0\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") "
Jul 10 20:14:48.340610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.340528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-util\") pod \"355e96c3-2a65-4e69-9818-c73870010ea0\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") "
Jul 10 20:14:48.340610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.340568    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bnsc\" (UniqueName: \"kubernetes.io/projected/355e96c3-2a65-4e69-9818-c73870010ea0-kube-api-access-9bnsc\") pod \"355e96c3-2a65-4e69-9818-c73870010ea0\" (UID: \"355e96c3-2a65-4e69-9818-c73870010ea0\") "
Jul 10 20:14:48.341034 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.340981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-bundle" (OuterVolumeSpecName: "bundle") pod "355e96c3-2a65-4e69-9818-c73870010ea0" (UID: "355e96c3-2a65-4e69-9818-c73870010ea0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:48.342753 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.342732    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/355e96c3-2a65-4e69-9818-c73870010ea0-kube-api-access-9bnsc" (OuterVolumeSpecName: "kube-api-access-9bnsc") pod "355e96c3-2a65-4e69-9818-c73870010ea0" (UID: "355e96c3-2a65-4e69-9818-c73870010ea0"). InnerVolumeSpecName "kube-api-access-9bnsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:14:48.345327 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.345300    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-util" (OuterVolumeSpecName: "util") pod "355e96c3-2a65-4e69-9818-c73870010ea0" (UID: "355e96c3-2a65-4e69-9818-c73870010ea0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:48.441761 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.441656    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-bundle\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:48.441761 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.441709    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/355e96c3-2a65-4e69-9818-c73870010ea0-util\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:48.441761 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:48.441721    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bnsc\" (UniqueName: \"kubernetes.io/projected/355e96c3-2a65-4e69-9818-c73870010ea0-kube-api-access-9bnsc\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:49.148404 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.148378    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls"
Jul 10 20:14:49.148863 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.148414    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls" event={"ID":"355e96c3-2a65-4e69-9818-c73870010ea0","Type":"ContainerDied","Data":"a915f3ea638daebdf3c877a9de5e0036acf3c703052f188fbf0de99a0b852421"}
Jul 10 20:14:49.148863 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.148489    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a915f3ea638daebdf3c877a9de5e0036acf3c703052f188fbf0de99a0b852421"
Jul 10 20:14:49.280941 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.280917    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:14:49.310312 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.310291    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:49.348628 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.348592    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-bundle\") pod \"78375b72-580e-4f47-a6f4-226c88ee9629\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") "
Jul 10 20:14:49.348829 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.348643    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-util\") pod \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") "
Jul 10 20:14:49.348829 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.348694    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-bundle\") pod \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") "
Jul 10 20:14:49.348829 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.348717    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-util\") pod \"78375b72-580e-4f47-a6f4-226c88ee9629\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") "
Jul 10 20:14:49.348829 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.348736    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkfrz\" (UniqueName: \"kubernetes.io/projected/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-kube-api-access-nkfrz\") pod \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\" (UID: \"9b99e262-2fe2-46ac-ad69-7b8674fad8b5\") "
Jul 10 20:14:49.348829 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.348763    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ft7g\" (UniqueName: \"kubernetes.io/projected/78375b72-580e-4f47-a6f4-226c88ee9629-kube-api-access-2ft7g\") pod \"78375b72-580e-4f47-a6f4-226c88ee9629\" (UID: \"78375b72-580e-4f47-a6f4-226c88ee9629\") "
Jul 10 20:14:49.349233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.349204    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-bundle" (OuterVolumeSpecName: "bundle") pod "9b99e262-2fe2-46ac-ad69-7b8674fad8b5" (UID: "9b99e262-2fe2-46ac-ad69-7b8674fad8b5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:49.349233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.349213    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-bundle" (OuterVolumeSpecName: "bundle") pod "78375b72-580e-4f47-a6f4-226c88ee9629" (UID: "78375b72-580e-4f47-a6f4-226c88ee9629"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:49.350860 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.350830    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-kube-api-access-nkfrz" (OuterVolumeSpecName: "kube-api-access-nkfrz") pod "9b99e262-2fe2-46ac-ad69-7b8674fad8b5" (UID: "9b99e262-2fe2-46ac-ad69-7b8674fad8b5"). InnerVolumeSpecName "kube-api-access-nkfrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:14:49.351257 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.351223    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78375b72-580e-4f47-a6f4-226c88ee9629-kube-api-access-2ft7g" (OuterVolumeSpecName: "kube-api-access-2ft7g") pod "78375b72-580e-4f47-a6f4-226c88ee9629" (UID: "78375b72-580e-4f47-a6f4-226c88ee9629"). InnerVolumeSpecName "kube-api-access-2ft7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:14:49.354832 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.354810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-util" (OuterVolumeSpecName: "util") pod "78375b72-580e-4f47-a6f4-226c88ee9629" (UID: "78375b72-580e-4f47-a6f4-226c88ee9629"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:49.357385 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.357345    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-util" (OuterVolumeSpecName: "util") pod "9b99e262-2fe2-46ac-ad69-7b8674fad8b5" (UID: "9b99e262-2fe2-46ac-ad69-7b8674fad8b5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:14:49.449526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.449439    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-bundle\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:49.449526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.449471    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-util\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:49.449526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.449483    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkfrz\" (UniqueName: \"kubernetes.io/projected/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-kube-api-access-nkfrz\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:49.449526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.449498    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ft7g\" (UniqueName: \"kubernetes.io/projected/78375b72-580e-4f47-a6f4-226c88ee9629-kube-api-access-2ft7g\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:49.449526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.449510    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78375b72-580e-4f47-a6f4-226c88ee9629-bundle\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:49.449526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:49.449521    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b99e262-2fe2-46ac-ad69-7b8674fad8b5-util\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:14:50.155277 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:50.155241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc" event={"ID":"78375b72-580e-4f47-a6f4-226c88ee9629","Type":"ContainerDied","Data":"bd6decca26ccee167b19366c9835d7d2d2c7b85f7388685f532affcf04ae6b0b"}
Jul 10 20:14:50.155736 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:50.155287    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd6decca26ccee167b19366c9835d7d2d2c7b85f7388685f532affcf04ae6b0b"
Jul 10 20:14:50.155736 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:50.155253    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc"
Jul 10 20:14:50.157058 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:50.157036    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll" event={"ID":"9b99e262-2fe2-46ac-ad69-7b8674fad8b5","Type":"ContainerDied","Data":"49e678bd9e776ab358e1e58e2ba0258381f44a58e17666fd32444b175e1f98b9"}
Jul 10 20:14:50.157148 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:50.157075    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49e678bd9e776ab358e1e58e2ba0258381f44a58e17666fd32444b175e1f98b9"
Jul 10 20:14:50.157148 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:14:50.157081    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll"
Jul 10 20:15:31.768328 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768291    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"]
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768625    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf734a4-c170-4437-8a61-a23790b66e00" containerName="pull"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768638    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf734a4-c170-4437-8a61-a23790b66e00" containerName="pull"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768646    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerName="pull"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768652    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerName="pull"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768664    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768685    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768695    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="355e96c3-2a65-4e69-9818-c73870010ea0" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768701    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="355e96c3-2a65-4e69-9818-c73870010ea0" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768708    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78375b72-580e-4f47-a6f4-226c88ee9629" containerName="extract"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768712    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78375b72-580e-4f47-a6f4-226c88ee9629" containerName="extract"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768718    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="355e96c3-2a65-4e69-9818-c73870010ea0" containerName="pull"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768723    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="355e96c3-2a65-4e69-9818-c73870010ea0" containerName="pull"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768730    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78375b72-580e-4f47-a6f4-226c88ee9629" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768735    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78375b72-580e-4f47-a6f4-226c88ee9629" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768744    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf734a4-c170-4437-8a61-a23790b66e00" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768749    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf734a4-c170-4437-8a61-a23790b66e00" containerName="util"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768759    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerName="extract"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768767    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerName="extract"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768777    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="355e96c3-2a65-4e69-9818-c73870010ea0" containerName="extract"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768785    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="355e96c3-2a65-4e69-9818-c73870010ea0" containerName="extract"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768791    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78375b72-580e-4f47-a6f4-226c88ee9629" containerName="pull"
Jul 10 20:15:31.768790 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768798    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="78375b72-580e-4f47-a6f4-226c88ee9629" containerName="pull"
Jul 10 20:15:31.769692 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768805    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf734a4-c170-4437-8a61-a23790b66e00" containerName="extract"
Jul 10 20:15:31.769692 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768810    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf734a4-c170-4437-8a61-a23790b66e00" containerName="extract"
Jul 10 20:15:31.769692 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768875    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b99e262-2fe2-46ac-ad69-7b8674fad8b5" containerName="extract"
Jul 10 20:15:31.769692 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768888    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf734a4-c170-4437-8a61-a23790b66e00" containerName="extract"
Jul 10 20:15:31.769692 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768899    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="355e96c3-2a65-4e69-9818-c73870010ea0" containerName="extract"
Jul 10 20:15:31.769692 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.768906    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="78375b72-580e-4f47-a6f4-226c88ee9629" containerName="extract"
Jul 10 20:15:31.771906 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.771889    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:31.774699 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.774652    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 20:15:31.774817 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.774718    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 20:15:31.774983 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.774968    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z247m\""
Jul 10 20:15:31.787133 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.787106    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"]
Jul 10 20:15:31.801329 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.801301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43deec5b-b235-44e5-baca-28b89b2edffa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:31.801329 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.801335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2fcd\" (UniqueName: \"kubernetes.io/projected/43deec5b-b235-44e5-baca-28b89b2edffa-kube-api-access-q2fcd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:31.902922 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.902882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43deec5b-b235-44e5-baca-28b89b2edffa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:31.902922 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.902925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2fcd\" (UniqueName: \"kubernetes.io/projected/43deec5b-b235-44e5-baca-28b89b2edffa-kube-api-access-q2fcd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:31.903326 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.903304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43deec5b-b235-44e5-baca-28b89b2edffa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:31.911640 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:31.911608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2fcd\" (UniqueName: \"kubernetes.io/projected/43deec5b-b235-44e5-baca-28b89b2edffa-kube-api-access-q2fcd\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:32.082493 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.082404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:32.217881 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.217854    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"]
Jul 10 20:15:32.220294 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:15:32.220256    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43deec5b_b235_44e5_baca_28b89b2edffa.slice/crio-731190a5905d9838f012c30f7b6acdb281c85c0be7aed484a71425c81644f1bc WatchSource:0}: Error finding container 731190a5905d9838f012c30f7b6acdb281c85c0be7aed484a71425c81644f1bc: Status 404 returned error can't find the container with id 731190a5905d9838f012c30f7b6acdb281c85c0be7aed484a71425c81644f1bc
Jul 10 20:15:32.353763 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.353681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl" event={"ID":"43deec5b-b235-44e5-baca-28b89b2edffa","Type":"ContainerStarted","Data":"731190a5905d9838f012c30f7b6acdb281c85c0be7aed484a71425c81644f1bc"}
Jul 10 20:15:32.724502 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.724431    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"]
Jul 10 20:15:32.728465 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.728440    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"]
Jul 10 20:15:32.744823 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.744790    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"]
Jul 10 20:15:32.747912 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.747890    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:32.764750 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.764717    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"]
Jul 10 20:15:32.809210 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.809112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgmt9\" (UniqueName: \"kubernetes.io/projected/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-kube-api-access-lgmt9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8lw4z\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:32.809210 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.809163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8lw4z\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:32.910047 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.910018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgmt9\" (UniqueName: \"kubernetes.io/projected/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-kube-api-access-lgmt9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8lw4z\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:32.910214 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.910057    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8lw4z\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:32.910487 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.910448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8lw4z\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:32.918246 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:32.918217    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgmt9\" (UniqueName: \"kubernetes.io/projected/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-kube-api-access-lgmt9\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8lw4z\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:33.061404 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:33.061362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:33.295477 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:33.295446    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"]
Jul 10 20:15:33.299063 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:15:33.299033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb7b21e_9154_41b9_8b16_6eb1ce3ac9e0.slice/crio-debb2b1eed077c022491328d207485b6e9e2ffe8da9e5319e62b97800838fa87 WatchSource:0}: Error finding container debb2b1eed077c022491328d207485b6e9e2ffe8da9e5319e62b97800838fa87: Status 404 returned error can't find the container with id debb2b1eed077c022491328d207485b6e9e2ffe8da9e5319e62b97800838fa87
Jul 10 20:15:33.368565 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:33.368478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z" event={"ID":"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0","Type":"ContainerStarted","Data":"debb2b1eed077c022491328d207485b6e9e2ffe8da9e5319e62b97800838fa87"}
Jul 10 20:15:37.394435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.394330    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" containerName="manager" containerID="cri-o://78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305" gracePeriod=2
Jul 10 20:15:37.396034 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.396005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z" event={"ID":"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0","Type":"ContainerStarted","Data":"24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a"}
Jul 10 20:15:37.396168 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.396150    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:15:37.416709 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.416643    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z" podStartSLOduration=1.634284597 podStartE2EDuration="5.416629721s" totalImagesPullingTime="3.782345124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:15:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:15:37.415245694 +0000 UTC m=+424.641012624" watchObservedRunningTime="2026-07-10 20:15:37.416629721 +0000 UTC m=+424.642396650"
Jul 10 20:15:37.636602 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.636578    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:37.638759 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:15:37.638732    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:37.646627 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.646572    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2fcd\" (UniqueName: \"kubernetes.io/projected/43deec5b-b235-44e5-baca-28b89b2edffa-kube-api-access-q2fcd\") pod \"43deec5b-b235-44e5-baca-28b89b2edffa\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") "
Jul 10 20:15:37.646744 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.646649    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43deec5b-b235-44e5-baca-28b89b2edffa-extensions-socket-volume\") pod \"43deec5b-b235-44e5-baca-28b89b2edffa\" (UID: \"43deec5b-b235-44e5-baca-28b89b2edffa\") "
Jul 10 20:15:37.646905 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.646880    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43deec5b-b235-44e5-baca-28b89b2edffa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "43deec5b-b235-44e5-baca-28b89b2edffa" (UID: "43deec5b-b235-44e5-baca-28b89b2edffa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:15:37.648535 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.648511    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43deec5b-b235-44e5-baca-28b89b2edffa-kube-api-access-q2fcd" (OuterVolumeSpecName: "kube-api-access-q2fcd") pod "43deec5b-b235-44e5-baca-28b89b2edffa" (UID: "43deec5b-b235-44e5-baca-28b89b2edffa"). InnerVolumeSpecName "kube-api-access-q2fcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:15:37.747605 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.747566    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2fcd\" (UniqueName: \"kubernetes.io/projected/43deec5b-b235-44e5-baca-28b89b2edffa-kube-api-access-q2fcd\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:15:37.747605 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:37.747598    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43deec5b-b235-44e5-baca-28b89b2edffa-extensions-socket-volume\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:15:38.402777 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:38.402690    2580 generic.go:356] "Generic (PLEG): container finished" podID="43deec5b-b235-44e5-baca-28b89b2edffa" containerID="78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305" exitCode=2
Jul 10 20:15:38.402777 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:38.402737    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:38.403258 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:38.402779    2580 scope.go:117] "RemoveContainer" containerID="78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305"
Jul 10 20:15:38.405022 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:15:38.404998    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:38.412642 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:38.412616    2580 scope.go:117] "RemoveContainer" containerID="78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305"
Jul 10 20:15:38.412771 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:15:38.412740    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-v4trl\" is forbidden: User \"system:node:ip-10-0-133-106.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-106.ec2.internal' and this object" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-v4trl"
Jul 10 20:15:38.412940 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:15:38.412922    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305\": container with ID starting with 78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305 not found: ID does not exist" containerID="78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305"
Jul 10 20:15:38.412980 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:38.412950    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305"} err="failed to get container status \"78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305\": rpc error: code = NotFound desc = could not find container \"78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305\": container with ID starting with 78c886e6ba4d9252ae720e89a4f7bc3a5843e55ddd9a3a3f9036aba32aa56305 not found: ID does not exist"
Jul 10 20:15:39.280492 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:39.280451    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" path="/var/lib/kubelet/pods/43deec5b-b235-44e5-baca-28b89b2edffa/volumes"
Jul 10 20:15:48.405371 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:15:48.405339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:16:01.098558 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.098521    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"]
Jul 10 20:16:01.099097 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.098726    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z" podUID="2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" containerName="manager" containerID="cri-o://24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a" gracePeriod=10
Jul 10 20:16:01.349724 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.349655    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:16:01.438817 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.438778    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgmt9\" (UniqueName: \"kubernetes.io/projected/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-kube-api-access-lgmt9\") pod \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") "
Jul 10 20:16:01.439014 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.438826    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-extensions-socket-volume\") pod \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\" (UID: \"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0\") "
Jul 10 20:16:01.439319 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.439295    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" (UID: "2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:16:01.440897 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.440875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-kube-api-access-lgmt9" (OuterVolumeSpecName: "kube-api-access-lgmt9") pod "2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" (UID: "2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0"). InnerVolumeSpecName "kube-api-access-lgmt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:16:01.516273 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.516240    2580 generic.go:356] "Generic (PLEG): container finished" podID="2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" containerID="24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a" exitCode=0
Jul 10 20:16:01.516429 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.516283    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"
Jul 10 20:16:01.516429 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.516322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z" event={"ID":"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0","Type":"ContainerDied","Data":"24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a"}
Jul 10 20:16:01.516429 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.516370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z" event={"ID":"2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0","Type":"ContainerDied","Data":"debb2b1eed077c022491328d207485b6e9e2ffe8da9e5319e62b97800838fa87"}
Jul 10 20:16:01.516429 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.516393    2580 scope.go:117] "RemoveContainer" containerID="24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a"
Jul 10 20:16:01.525160 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.525142    2580 scope.go:117] "RemoveContainer" containerID="24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a"
Jul 10 20:16:01.525446 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:16:01.525428    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a\": container with ID starting with 24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a not found: ID does not exist" containerID="24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a"
Jul 10 20:16:01.525496 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.525456    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a"} err="failed to get container status \"24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a\": rpc error: code = NotFound desc = could not find container \"24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a\": container with ID starting with 24fdcf633469ef5ca72916a15b5829c67b5d40f764589a9ac30d134f3f9ed78a not found: ID does not exist"
Jul 10 20:16:01.538352 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.538323    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"]
Jul 10 20:16:01.539392 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.539371    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgmt9\" (UniqueName: \"kubernetes.io/projected/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-kube-api-access-lgmt9\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:16:01.539442 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.539394    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0-extensions-socket-volume\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:16:01.544661 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:01.544637    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8lw4z"]
Jul 10 20:16:03.280117 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:03.280085    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" path="/var/lib/kubelet/pods/2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0/volumes"
Jul 10 20:16:20.825323 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.825284    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"]
Jul 10 20:16:20.825740 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.825591    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" containerName="manager"
Jul 10 20:16:20.825740 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.825602    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" containerName="manager"
Jul 10 20:16:20.825740 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.825614    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" containerName="manager"
Jul 10 20:16:20.825740 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.825620    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" containerName="manager"
Jul 10 20:16:20.825740 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.825664    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="43deec5b-b235-44e5-baca-28b89b2edffa" containerName="manager"
Jul 10 20:16:20.825740 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.825686    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fb7b21e-9154-41b9-8b16-6eb1ce3ac9e0" containerName="manager"
Jul 10 20:16:20.829234 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.829214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:20.831760 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.831739    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 20:16:20.831874 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.831770    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 10 20:16:20.831874 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.831801    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 20:16:20.832726 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.832707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7rtwr\""
Jul 10 20:16:20.838445 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.838424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"]
Jul 10 20:16:20.914006 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.913978    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66x4n\" (UniqueName: \"kubernetes.io/projected/749b489c-6528-49ee-8ad3-3a2e57ceee1f-kube-api-access-66x4n\") pod \"limitador-limitador-64c94fd64f-xw9qd\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:20.914174 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.914030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b489c-6528-49ee-8ad3-3a2e57ceee1f-config-file\") pod \"limitador-limitador-64c94fd64f-xw9qd\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:20.926635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:20.926607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"]
Jul 10 20:16:21.015174 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:21.015131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b489c-6528-49ee-8ad3-3a2e57ceee1f-config-file\") pod \"limitador-limitador-64c94fd64f-xw9qd\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:21.015363 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:21.015235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66x4n\" (UniqueName: \"kubernetes.io/projected/749b489c-6528-49ee-8ad3-3a2e57ceee1f-kube-api-access-66x4n\") pod \"limitador-limitador-64c94fd64f-xw9qd\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:21.015898 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:21.015874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b489c-6528-49ee-8ad3-3a2e57ceee1f-config-file\") pod \"limitador-limitador-64c94fd64f-xw9qd\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:21.022903 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:21.022876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66x4n\" (UniqueName: \"kubernetes.io/projected/749b489c-6528-49ee-8ad3-3a2e57ceee1f-kube-api-access-66x4n\") pod \"limitador-limitador-64c94fd64f-xw9qd\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:21.140409 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:21.140331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:21.292087 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:21.292065    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"]
Jul 10 20:16:21.293956 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:16:21.293920    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749b489c_6528_49ee_8ad3_3a2e57ceee1f.slice/crio-74a93b3f3123e17ea2a4ce621937aa58da75a8bb82ce69751ca84dc0d4ba51c8 WatchSource:0}: Error finding container 74a93b3f3123e17ea2a4ce621937aa58da75a8bb82ce69751ca84dc0d4ba51c8: Status 404 returned error can't find the container with id 74a93b3f3123e17ea2a4ce621937aa58da75a8bb82ce69751ca84dc0d4ba51c8
Jul 10 20:16:21.618563 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:21.618524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd" event={"ID":"749b489c-6528-49ee-8ad3-3a2e57ceee1f","Type":"ContainerStarted","Data":"74a93b3f3123e17ea2a4ce621937aa58da75a8bb82ce69751ca84dc0d4ba51c8"}
Jul 10 20:16:25.646628 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:25.646594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd" event={"ID":"749b489c-6528-49ee-8ad3-3a2e57ceee1f","Type":"ContainerStarted","Data":"cb4adfd10701bf2792b035947e666f8c0e35d5a1bb8603e453d9c843246319f8"}
Jul 10 20:16:25.647045 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:25.646737    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:25.662650 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:25.662597    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd" podStartSLOduration=2.207605534 podStartE2EDuration="5.662583167s" totalImagesPullingTime="3.454977633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:16:25.660589492 +0000 UTC m=+472.886356424" watchObservedRunningTime="2026-07-10 20:16:25.662583167 +0000 UTC m=+472.888350095"
Jul 10 20:16:36.240632 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:36.240594    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"]
Jul 10 20:16:36.241114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:36.240821    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd" podUID="749b489c-6528-49ee-8ad3-3a2e57ceee1f" containerName="limitador" containerID="cri-o://cb4adfd10701bf2792b035947e666f8c0e35d5a1bb8603e453d9c843246319f8" gracePeriod=30
Jul 10 20:16:36.241532 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:36.241503    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:36.706001 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:36.705971    2580 generic.go:356] "Generic (PLEG): container finished" podID="749b489c-6528-49ee-8ad3-3a2e57ceee1f" containerID="cb4adfd10701bf2792b035947e666f8c0e35d5a1bb8603e453d9c843246319f8" exitCode=0
Jul 10 20:16:36.706169 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:36.706035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd" event={"ID":"749b489c-6528-49ee-8ad3-3a2e57ceee1f","Type":"ContainerDied","Data":"cb4adfd10701bf2792b035947e666f8c0e35d5a1bb8603e453d9c843246319f8"}
Jul 10 20:16:37.184180 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.184153    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:37.250751 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.250720    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66x4n\" (UniqueName: \"kubernetes.io/projected/749b489c-6528-49ee-8ad3-3a2e57ceee1f-kube-api-access-66x4n\") pod \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") "
Jul 10 20:16:37.250751 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.250752    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b489c-6528-49ee-8ad3-3a2e57ceee1f-config-file\") pod \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\" (UID: \"749b489c-6528-49ee-8ad3-3a2e57ceee1f\") "
Jul 10 20:16:37.251160 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.251137    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/749b489c-6528-49ee-8ad3-3a2e57ceee1f-config-file" (OuterVolumeSpecName: "config-file") pod "749b489c-6528-49ee-8ad3-3a2e57ceee1f" (UID: "749b489c-6528-49ee-8ad3-3a2e57ceee1f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:16:37.252807 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.252787    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749b489c-6528-49ee-8ad3-3a2e57ceee1f-kube-api-access-66x4n" (OuterVolumeSpecName: "kube-api-access-66x4n") pod "749b489c-6528-49ee-8ad3-3a2e57ceee1f" (UID: "749b489c-6528-49ee-8ad3-3a2e57ceee1f"). InnerVolumeSpecName "kube-api-access-66x4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:16:37.351661 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.351568    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66x4n\" (UniqueName: \"kubernetes.io/projected/749b489c-6528-49ee-8ad3-3a2e57ceee1f-kube-api-access-66x4n\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:16:37.351661 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.351599    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/749b489c-6528-49ee-8ad3-3a2e57ceee1f-config-file\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:16:37.711844 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.711768    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"
Jul 10 20:16:37.711844 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.711785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xw9qd" event={"ID":"749b489c-6528-49ee-8ad3-3a2e57ceee1f","Type":"ContainerDied","Data":"74a93b3f3123e17ea2a4ce621937aa58da75a8bb82ce69751ca84dc0d4ba51c8"}
Jul 10 20:16:37.711844 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.711843    2580 scope.go:117] "RemoveContainer" containerID="cb4adfd10701bf2792b035947e666f8c0e35d5a1bb8603e453d9c843246319f8"
Jul 10 20:16:37.728278 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.728251    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"]
Jul 10 20:16:37.730430 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:37.730406    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xw9qd"]
Jul 10 20:16:39.034251 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.034222    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"]
Jul 10 20:16:39.034608 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.034540    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="749b489c-6528-49ee-8ad3-3a2e57ceee1f" containerName="limitador"
Jul 10 20:16:39.034608 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.034551    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="749b489c-6528-49ee-8ad3-3a2e57ceee1f" containerName="limitador"
Jul 10 20:16:39.034712 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.034613    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="749b489c-6528-49ee-8ad3-3a2e57ceee1f" containerName="limitador"
Jul 10 20:16:39.038938 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.038916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.042406 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.042387    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 20:16:39.043489 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.043472    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 10 20:16:39.043489 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.043483    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-tfsfv\""
Jul 10 20:16:39.043648 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.043477    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 20:16:39.052588 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.052564    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"]
Jul 10 20:16:39.065725 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.065698    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4qlq\" (UniqueName: \"kubernetes.io/projected/e46b6254-9c5f-48ad-a66c-98344d3c990d-kube-api-access-q4qlq\") pod \"postgres-69fb4594cf-xk7v9\" (UID: \"e46b6254-9c5f-48ad-a66c-98344d3c990d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.065846 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.065730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e46b6254-9c5f-48ad-a66c-98344d3c990d-data\") pod \"postgres-69fb4594cf-xk7v9\" (UID: \"e46b6254-9c5f-48ad-a66c-98344d3c990d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.167155 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.167119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4qlq\" (UniqueName: \"kubernetes.io/projected/e46b6254-9c5f-48ad-a66c-98344d3c990d-kube-api-access-q4qlq\") pod \"postgres-69fb4594cf-xk7v9\" (UID: \"e46b6254-9c5f-48ad-a66c-98344d3c990d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.167155 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.167158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e46b6254-9c5f-48ad-a66c-98344d3c990d-data\") pod \"postgres-69fb4594cf-xk7v9\" (UID: \"e46b6254-9c5f-48ad-a66c-98344d3c990d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.167587 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.167567    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e46b6254-9c5f-48ad-a66c-98344d3c990d-data\") pod \"postgres-69fb4594cf-xk7v9\" (UID: \"e46b6254-9c5f-48ad-a66c-98344d3c990d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.176100 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.176078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4qlq\" (UniqueName: \"kubernetes.io/projected/e46b6254-9c5f-48ad-a66c-98344d3c990d-kube-api-access-q4qlq\") pod \"postgres-69fb4594cf-xk7v9\" (UID: \"e46b6254-9c5f-48ad-a66c-98344d3c990d\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.280403 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.280369    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="749b489c-6528-49ee-8ad3-3a2e57ceee1f" path="/var/lib/kubelet/pods/749b489c-6528-49ee-8ad3-3a2e57ceee1f/volumes"
Jul 10 20:16:39.348809 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.348719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:39.516213 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.516135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"]
Jul 10 20:16:39.518540 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:16:39.518501    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46b6254_9c5f_48ad_a66c_98344d3c990d.slice/crio-b419288fa4d408408f2bfcc2e18badbca7b4400fc163d764562e4270aa997f58 WatchSource:0}: Error finding container b419288fa4d408408f2bfcc2e18badbca7b4400fc163d764562e4270aa997f58: Status 404 returned error can't find the container with id b419288fa4d408408f2bfcc2e18badbca7b4400fc163d764562e4270aa997f58
Jul 10 20:16:39.724532 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:39.724450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9" event={"ID":"e46b6254-9c5f-48ad-a66c-98344d3c990d","Type":"ContainerStarted","Data":"b419288fa4d408408f2bfcc2e18badbca7b4400fc163d764562e4270aa997f58"}
Jul 10 20:16:45.768371 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:45.768334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9" event={"ID":"e46b6254-9c5f-48ad-a66c-98344d3c990d","Type":"ContainerStarted","Data":"398a1a3ccb837f632bdf7dcdfd7a5b1e7e46fed70c438fcc6dec81ee1716f5da"}
Jul 10 20:16:45.768816 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:45.768416    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:45.784879 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:45.784828    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9" podStartSLOduration=1.5707339519999999 podStartE2EDuration="6.784812533s" totalImagesPullingTime="5.214078581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:16:45.782495426 +0000 UTC m=+493.008262358" watchObservedRunningTime="2026-07-10 20:16:45.784812533 +0000 UTC m=+493.010579538"
Jul 10 20:16:51.801995 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:51.801966    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xk7v9"
Jul 10 20:16:58.804983 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.804944    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5dd9c7cfb-z54ww"]
Jul 10 20:16:58.953917 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.953874    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dd9c7cfb-z54ww"]
Jul 10 20:16:58.954106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.954004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:58.956437 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.956415    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 20:16:58.956437 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.956418    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 20:16:58.956614 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.956473    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vtb2p\""
Jul 10 20:16:58.975655 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.975626    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-sftdl"]
Jul 10 20:16:58.983579 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.983558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:58.989879 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:58.989852    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-sftdl"]
Jul 10 20:16:59.047378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.047342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert\") pod \"maas-controller-56c7cffc4c-sftdl\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") " pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:59.047378 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.047378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d6sq\" (UniqueName: \"kubernetes.io/projected/9788e735-79a2-4719-bdb6-47bd280a3b48-kube-api-access-9d6sq\") pod \"maas-controller-56c7cffc4c-sftdl\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") " pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:59.047585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.047408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqgqx\" (UniqueName: \"kubernetes.io/projected/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-kube-api-access-xqgqx\") pod \"maas-controller-5dd9c7cfb-z54ww\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") " pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.047585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.047522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert\") pod \"maas-controller-5dd9c7cfb-z54ww\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") " pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.148946 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.148861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert\") pod \"maas-controller-5dd9c7cfb-z54ww\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") " pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.148946 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.148930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert\") pod \"maas-controller-56c7cffc4c-sftdl\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") " pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:59.148946 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.148947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d6sq\" (UniqueName: \"kubernetes.io/projected/9788e735-79a2-4719-bdb6-47bd280a3b48-kube-api-access-9d6sq\") pod \"maas-controller-56c7cffc4c-sftdl\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") " pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:59.149161 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.148967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqgqx\" (UniqueName: \"kubernetes.io/projected/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-kube-api-access-xqgqx\") pod \"maas-controller-5dd9c7cfb-z54ww\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") " pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.149161 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:16:59.149033    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 10 20:16:59.149161 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:16:59.149061    2580 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 10 20:16:59.149161 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:16:59.149125    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert podName:9788e735-79a2-4719-bdb6-47bd280a3b48 nodeName:}" failed. No retries permitted until 2026-07-10 20:16:59.649102123 +0000 UTC m=+506.874869032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert") pod "maas-controller-56c7cffc4c-sftdl" (UID: "9788e735-79a2-4719-bdb6-47bd280a3b48") : secret "maas-controller-webhook-cert" not found
Jul 10 20:16:59.149161 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:16:59.149145    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert podName:c85c09f0-a5eb-4c25-a988-fa62d70e1e63 nodeName:}" failed. No retries permitted until 2026-07-10 20:16:59.649136587 +0000 UTC m=+506.874903494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert") pod "maas-controller-5dd9c7cfb-z54ww" (UID: "c85c09f0-a5eb-4c25-a988-fa62d70e1e63") : secret "maas-controller-webhook-cert" not found
Jul 10 20:16:59.157953 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.157926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d6sq\" (UniqueName: \"kubernetes.io/projected/9788e735-79a2-4719-bdb6-47bd280a3b48-kube-api-access-9d6sq\") pod \"maas-controller-56c7cffc4c-sftdl\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") " pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:59.158073 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.158000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqgqx\" (UniqueName: \"kubernetes.io/projected/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-kube-api-access-xqgqx\") pod \"maas-controller-5dd9c7cfb-z54ww\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") " pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.654495 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.654456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert\") pod \"maas-controller-5dd9c7cfb-z54ww\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") " pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.654773 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.654602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert\") pod \"maas-controller-56c7cffc4c-sftdl\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") " pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:59.656834 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.656806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert\") pod \"maas-controller-56c7cffc4c-sftdl\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") " pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:16:59.656941 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.656868    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert\") pod \"maas-controller-5dd9c7cfb-z54ww\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") " pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.864287 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.864253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:16:59.894052 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:16:59.894024    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:17:00.013708 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:00.013660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5dd9c7cfb-z54ww"]
Jul 10 20:17:00.016094 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:17:00.016061    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85c09f0_a5eb_4c25_a988_fa62d70e1e63.slice/crio-44f3e2c66e57572a8910294e51f46be0bb9248d9b3ae7785d7ce4fdd2055bb6e WatchSource:0}: Error finding container 44f3e2c66e57572a8910294e51f46be0bb9248d9b3ae7785d7ce4fdd2055bb6e: Status 404 returned error can't find the container with id 44f3e2c66e57572a8910294e51f46be0bb9248d9b3ae7785d7ce4fdd2055bb6e
Jul 10 20:17:00.038489 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:00.038441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-sftdl"]
Jul 10 20:17:00.040502 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:17:00.040478    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9788e735_79a2_4719_bdb6_47bd280a3b48.slice/crio-7066eb613c50e151206bf52b6b8ed4c51f89ab48c8e57e0ac23e149122ebdbd1 WatchSource:0}: Error finding container 7066eb613c50e151206bf52b6b8ed4c51f89ab48c8e57e0ac23e149122ebdbd1: Status 404 returned error can't find the container with id 7066eb613c50e151206bf52b6b8ed4c51f89ab48c8e57e0ac23e149122ebdbd1
Jul 10 20:17:00.854916 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:00.854834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww" event={"ID":"c85c09f0-a5eb-4c25-a988-fa62d70e1e63","Type":"ContainerStarted","Data":"44f3e2c66e57572a8910294e51f46be0bb9248d9b3ae7785d7ce4fdd2055bb6e"}
Jul 10 20:17:00.856444 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:00.856413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c7cffc4c-sftdl" event={"ID":"9788e735-79a2-4719-bdb6-47bd280a3b48","Type":"ContainerStarted","Data":"7066eb613c50e151206bf52b6b8ed4c51f89ab48c8e57e0ac23e149122ebdbd1"}
Jul 10 20:17:02.869026 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:02.868970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c7cffc4c-sftdl" event={"ID":"9788e735-79a2-4719-bdb6-47bd280a3b48","Type":"ContainerStarted","Data":"49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72"}
Jul 10 20:17:02.869478 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:02.869044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:17:02.870732 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:02.870708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww" event={"ID":"c85c09f0-a5eb-4c25-a988-fa62d70e1e63","Type":"ContainerStarted","Data":"d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5"}
Jul 10 20:17:02.870848 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:02.870815    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:17:02.891315 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:02.891259    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56c7cffc4c-sftdl" podStartSLOduration=2.832087358 podStartE2EDuration="4.891241175s" totalImagesPullingTime="2.059153817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:17:02.888928786 +0000 UTC m=+510.114695730" watchObservedRunningTime="2026-07-10 20:17:02.891241175 +0000 UTC m=+510.117008103"
Jul 10 20:17:02.908561 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:02.908508    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww" podStartSLOduration=2.830260666 podStartE2EDuration="4.908490458s" totalImagesPullingTime="2.078229792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:17:02.906620221 +0000 UTC m=+510.132387151" watchObservedRunningTime="2026-07-10 20:17:02.908490458 +0000 UTC m=+510.134257386"
Jul 10 20:17:13.879739 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:13.879703    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:17:13.880203 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:13.879802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:17:13.937389 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:13.937092    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dd9c7cfb-z54ww"]
Jul 10 20:17:13.937389 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:13.937322    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww" podUID="c85c09f0-a5eb-4c25-a988-fa62d70e1e63" containerName="manager" containerID="cri-o://d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5" gracePeriod=30
Jul 10 20:17:14.180169 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.180147    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:17:14.282885 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.282848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqgqx\" (UniqueName: \"kubernetes.io/projected/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-kube-api-access-xqgqx\") pod \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") "
Jul 10 20:17:14.283079 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.282902    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert\") pod \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\" (UID: \"c85c09f0-a5eb-4c25-a988-fa62d70e1e63\") "
Jul 10 20:17:14.285077 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.285048    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c85c09f0-a5eb-4c25-a988-fa62d70e1e63" (UID: "c85c09f0-a5eb-4c25-a988-fa62d70e1e63"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:17:14.285188 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.285152    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-kube-api-access-xqgqx" (OuterVolumeSpecName: "kube-api-access-xqgqx") pod "c85c09f0-a5eb-4c25-a988-fa62d70e1e63" (UID: "c85c09f0-a5eb-4c25-a988-fa62d70e1e63"). InnerVolumeSpecName "kube-api-access-xqgqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:17:14.384141 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.384100    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqgqx\" (UniqueName: \"kubernetes.io/projected/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-kube-api-access-xqgqx\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:17:14.384141 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.384144    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c85c09f0-a5eb-4c25-a988-fa62d70e1e63-webhook-cert\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:17:14.930522 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.930485    2580 generic.go:356] "Generic (PLEG): container finished" podID="c85c09f0-a5eb-4c25-a988-fa62d70e1e63" containerID="d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5" exitCode=0
Jul 10 20:17:14.930929 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.930540    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww"
Jul 10 20:17:14.930929 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.930575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww" event={"ID":"c85c09f0-a5eb-4c25-a988-fa62d70e1e63","Type":"ContainerDied","Data":"d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5"}
Jul 10 20:17:14.930929 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.930646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5dd9c7cfb-z54ww" event={"ID":"c85c09f0-a5eb-4c25-a988-fa62d70e1e63","Type":"ContainerDied","Data":"44f3e2c66e57572a8910294e51f46be0bb9248d9b3ae7785d7ce4fdd2055bb6e"}
Jul 10 20:17:14.930929 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.930679    2580 scope.go:117] "RemoveContainer" containerID="d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5"
Jul 10 20:17:14.939920 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.939761    2580 scope.go:117] "RemoveContainer" containerID="d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5"
Jul 10 20:17:14.940045 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:17:14.940025    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5\": container with ID starting with d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5 not found: ID does not exist" containerID="d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5"
Jul 10 20:17:14.940089 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.940055    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5"} err="failed to get container status \"d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5\": rpc error: code = NotFound desc = could not find container \"d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5\": container with ID starting with d00d742882556851d4b782356fba9b6c1371da0ea3d3d8a0d83f0f9a4908d5e5 not found: ID does not exist"
Jul 10 20:17:14.950733 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.950709    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5dd9c7cfb-z54ww"]
Jul 10 20:17:14.953783 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:14.953762    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5dd9c7cfb-z54ww"]
Jul 10 20:17:15.280264 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:15.280235    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c85c09f0-a5eb-4c25-a988-fa62d70e1e63" path="/var/lib/kubelet/pods/c85c09f0-a5eb-4c25-a988-fa62d70e1e63/volumes"
Jul 10 20:17:34.287709 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.287661    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"]
Jul 10 20:17:34.288114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.287994    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c85c09f0-a5eb-4c25-a988-fa62d70e1e63" containerName="manager"
Jul 10 20:17:34.288114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.288004    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c85c09f0-a5eb-4c25-a988-fa62d70e1e63" containerName="manager"
Jul 10 20:17:34.288114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.288066    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c85c09f0-a5eb-4c25-a988-fa62d70e1e63" containerName="manager"
Jul 10 20:17:34.292753 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.292722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.298373 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.298349    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 10 20:17:34.298513 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.298394    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-m4qfv\""
Jul 10 20:17:34.303154 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.303132    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"]
Jul 10 20:17:34.344598 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.344558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgtdr\" (UniqueName: \"kubernetes.io/projected/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-kube-api-access-rgtdr\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.344768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.344618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-maas-api-tls\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.344768 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.344739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-openshift-service-ca\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.445771 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.445737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-maas-api-tls\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.445934 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.445791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-openshift-service-ca\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.445934 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.445826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgtdr\" (UniqueName: \"kubernetes.io/projected/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-kube-api-access-rgtdr\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.446485 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.446459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-openshift-service-ca\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.448207 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.448186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-maas-api-tls\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.453715 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.453691    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgtdr\" (UniqueName: \"kubernetes.io/projected/772ffaa3-d62c-4fab-b85e-fe7fc0d03526-kube-api-access-rgtdr\") pod \"maas-api-fdf5c9f6f-lshj4\" (UID: \"772ffaa3-d62c-4fab-b85e-fe7fc0d03526\") " pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.603382 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.603289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:34.741153 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.741122    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"]
Jul 10 20:17:34.745508 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.745483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:34.747124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.747027    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66e1374-a44f-4dcd-a680-73a4c599fab7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-dwd4b\" (UID: \"b66e1374-a44f-4dcd-a680-73a4c599fab7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:34.747124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.747092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgss7\" (UniqueName: \"kubernetes.io/projected/b66e1374-a44f-4dcd-a680-73a4c599fab7-kube-api-access-hgss7\") pod \"payload-pre-processing-74584bb9f8-dwd4b\" (UID: \"b66e1374-a44f-4dcd-a680-73a4c599fab7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:34.747774 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.747751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"]
Jul 10 20:17:34.747774 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.747770    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 10 20:17:34.749884 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:17:34.749839    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772ffaa3_d62c_4fab_b85e_fe7fc0d03526.slice/crio-a92c8c5b92513755ea64c73a03e80ef113a55d93d93052a4b23b13790d820ccb WatchSource:0}: Error finding container a92c8c5b92513755ea64c73a03e80ef113a55d93d93052a4b23b13790d820ccb: Status 404 returned error can't find the container with id a92c8c5b92513755ea64c73a03e80ef113a55d93d93052a4b23b13790d820ccb
Jul 10 20:17:34.756038 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.756016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"]
Jul 10 20:17:34.775874 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.775846    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-4c54d"]
Jul 10 20:17:34.779201 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.779180    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:34.781868 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.781849    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m9rbf\""
Jul 10 20:17:34.788094 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.788070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-4c54d"]
Jul 10 20:17:34.848442 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.848408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add741dc-4042-4fdc-8a77-2d024a3280d2-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-4c54d\" (UID: \"add741dc-4042-4fdc-8a77-2d024a3280d2\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:34.848597 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.848448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgss7\" (UniqueName: \"kubernetes.io/projected/b66e1374-a44f-4dcd-a680-73a4c599fab7-kube-api-access-hgss7\") pod \"payload-pre-processing-74584bb9f8-dwd4b\" (UID: \"b66e1374-a44f-4dcd-a680-73a4c599fab7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:34.848597 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.848473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx6jh\" (UniqueName: \"kubernetes.io/projected/add741dc-4042-4fdc-8a77-2d024a3280d2-kube-api-access-wx6jh\") pod \"payload-processing-f9b8d9b9b-4c54d\" (UID: \"add741dc-4042-4fdc-8a77-2d024a3280d2\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:34.848597 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.848530    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66e1374-a44f-4dcd-a680-73a4c599fab7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-dwd4b\" (UID: \"b66e1374-a44f-4dcd-a680-73a4c599fab7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:34.849205 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.849187    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b66e1374-a44f-4dcd-a680-73a4c599fab7-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-dwd4b\" (UID: \"b66e1374-a44f-4dcd-a680-73a4c599fab7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:34.858154 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.858093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgss7\" (UniqueName: \"kubernetes.io/projected/b66e1374-a44f-4dcd-a680-73a4c599fab7-kube-api-access-hgss7\") pod \"payload-pre-processing-74584bb9f8-dwd4b\" (UID: \"b66e1374-a44f-4dcd-a680-73a4c599fab7\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:34.949467 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.949426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add741dc-4042-4fdc-8a77-2d024a3280d2-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-4c54d\" (UID: \"add741dc-4042-4fdc-8a77-2d024a3280d2\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:34.949649 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.949495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx6jh\" (UniqueName: \"kubernetes.io/projected/add741dc-4042-4fdc-8a77-2d024a3280d2-kube-api-access-wx6jh\") pod \"payload-processing-f9b8d9b9b-4c54d\" (UID: \"add741dc-4042-4fdc-8a77-2d024a3280d2\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:34.950244 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.950224    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/add741dc-4042-4fdc-8a77-2d024a3280d2-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-4c54d\" (UID: \"add741dc-4042-4fdc-8a77-2d024a3280d2\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:34.957205 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:34.957173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx6jh\" (UniqueName: \"kubernetes.io/projected/add741dc-4042-4fdc-8a77-2d024a3280d2-kube-api-access-wx6jh\") pod \"payload-processing-f9b8d9b9b-4c54d\" (UID: \"add741dc-4042-4fdc-8a77-2d024a3280d2\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:35.034045 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:35.033998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4" event={"ID":"772ffaa3-d62c-4fab-b85e-fe7fc0d03526","Type":"ContainerStarted","Data":"a92c8c5b92513755ea64c73a03e80ef113a55d93d93052a4b23b13790d820ccb"}
Jul 10 20:17:35.070195 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:35.070165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:35.090728 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:35.090701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:35.212328 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:35.212300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"]
Jul 10 20:17:35.214620 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:17:35.214594    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66e1374_a44f_4dcd_a680_73a4c599fab7.slice/crio-4f6bce977729ac898075c43bf754c03d643f6af292c0d63f2377e6fe041df708 WatchSource:0}: Error finding container 4f6bce977729ac898075c43bf754c03d643f6af292c0d63f2377e6fe041df708: Status 404 returned error can't find the container with id 4f6bce977729ac898075c43bf754c03d643f6af292c0d63f2377e6fe041df708
Jul 10 20:17:35.230533 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:35.230510    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-4c54d"]
Jul 10 20:17:35.231450 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:17:35.231417    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd741dc_4042_4fdc_8a77_2d024a3280d2.slice/crio-745c53ddf3792c4e80f1b53211edc154231d883739b539e29b2d5498692eeb3f WatchSource:0}: Error finding container 745c53ddf3792c4e80f1b53211edc154231d883739b539e29b2d5498692eeb3f: Status 404 returned error can't find the container with id 745c53ddf3792c4e80f1b53211edc154231d883739b539e29b2d5498692eeb3f
Jul 10 20:17:36.049045 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:36.048879    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b" event={"ID":"b66e1374-a44f-4dcd-a680-73a4c599fab7","Type":"ContainerStarted","Data":"4f6bce977729ac898075c43bf754c03d643f6af292c0d63f2377e6fe041df708"}
Jul 10 20:17:36.050805 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:36.050769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d" event={"ID":"add741dc-4042-4fdc-8a77-2d024a3280d2","Type":"ContainerStarted","Data":"745c53ddf3792c4e80f1b53211edc154231d883739b539e29b2d5498692eeb3f"}
Jul 10 20:17:38.063653 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.063615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4" event={"ID":"772ffaa3-d62c-4fab-b85e-fe7fc0d03526","Type":"ContainerStarted","Data":"b50f94caf8486856756c2f31720fe69a206c5ca505e9f5d5172149842487f5b0"}
Jul 10 20:17:38.064115 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.063732    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:38.065189 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.065158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b" event={"ID":"b66e1374-a44f-4dcd-a680-73a4c599fab7","Type":"ContainerStarted","Data":"61c331412f2dadccb26943a5a15a23d6d9969ca9d2c833c73609c605fe1ad130"}
Jul 10 20:17:38.065320 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.065244    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:38.066713 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.066665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d" event={"ID":"add741dc-4042-4fdc-8a77-2d024a3280d2","Type":"ContainerStarted","Data":"0b43da8371f61471ed4e9ab69a06c8846fd4830a4e84428b6038bfdebec7b96d"}
Jul 10 20:17:38.066831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.066729    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:38.081840 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.081798    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4" podStartSLOduration=1.846943913 podStartE2EDuration="4.08178519s" totalImagesPullingTime="2.234841277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:17:38.080689875 +0000 UTC m=+545.306456799" watchObservedRunningTime="2026-07-10 20:17:38.08178519 +0000 UTC m=+545.307552175"
Jul 10 20:17:38.096003 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.095960    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b" podStartSLOduration=2.324090992 podStartE2EDuration="4.0959466s" totalImagesPullingTime="1.771855608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:17:38.093646515 +0000 UTC m=+545.319413445" watchObservedRunningTime="2026-07-10 20:17:38.0959466 +0000 UTC m=+545.321713529"
Jul 10 20:17:38.109313 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:38.109270    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d" podStartSLOduration=2.356509773 podStartE2EDuration="4.109256763s" totalImagesPullingTime="1.75274699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:17:38.108441627 +0000 UTC m=+545.334208555" watchObservedRunningTime="2026-07-10 20:17:38.109256763 +0000 UTC m=+545.335023693"
Jul 10 20:17:44.077318 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:44.077293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-fdf5c9f6f-lshj4"
Jul 10 20:17:49.074788 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:49.074751    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-4c54d"
Jul 10 20:17:49.075228 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:49.074911    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-dwd4b"
Jul 10 20:17:50.531423 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.531381    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"]
Jul 10 20:17:50.535205 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.535181    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.538865 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.538836    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 10 20:17:50.538985 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.538838    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 20:17:50.538985 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.538838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wfnt5\""
Jul 10 20:17:50.538985 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.538908    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 20:17:50.543232 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.543213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"]
Jul 10 20:17:50.589818 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.589789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.589989 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.589845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n2jl\" (UniqueName: \"kubernetes.io/projected/3806b8a5-3878-4896-9df1-070f3d9b6f3d-kube-api-access-5n2jl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.590032 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.589977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.590077 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.590034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.590127 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.590096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.590172 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.590147    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3806b8a5-3878-4896-9df1-070f3d9b6f3d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.690902 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.690862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n2jl\" (UniqueName: \"kubernetes.io/projected/3806b8a5-3878-4896-9df1-070f3d9b6f3d-kube-api-access-5n2jl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691052 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.690924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691052 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.690949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691052 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.690969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691052 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.690991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3806b8a5-3878-4896-9df1-070f3d9b6f3d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691052 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.691017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691418 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.691393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691418 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.691412    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.691493 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.691449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.693370 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.693354    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3806b8a5-3878-4896-9df1-070f3d9b6f3d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.693742 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.693720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3806b8a5-3878-4896-9df1-070f3d9b6f3d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.698926 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.698903    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n2jl\" (UniqueName: \"kubernetes.io/projected/3806b8a5-3878-4896-9df1-070f3d9b6f3d-kube-api-access-5n2jl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8\" (UID: \"3806b8a5-3878-4896-9df1-070f3d9b6f3d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.846252 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.846166    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:17:50.978331 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:50.978301    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"]
Jul 10 20:17:50.980085 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:17:50.980056    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3806b8a5_3878_4896_9df1_070f3d9b6f3d.slice/crio-4c2eac4dbfe74bca584ddc26aa6e2b0bad9042b728b8d3f8cc4c2cb2f3dd3e4d WatchSource:0}: Error finding container 4c2eac4dbfe74bca584ddc26aa6e2b0bad9042b728b8d3f8cc4c2cb2f3dd3e4d: Status 404 returned error can't find the container with id 4c2eac4dbfe74bca584ddc26aa6e2b0bad9042b728b8d3f8cc4c2cb2f3dd3e4d
Jul 10 20:17:51.137746 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:51.137650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8" event={"ID":"3806b8a5-3878-4896-9df1-070f3d9b6f3d","Type":"ContainerStarted","Data":"4c2eac4dbfe74bca584ddc26aa6e2b0bad9042b728b8d3f8cc4c2cb2f3dd3e4d"}
Jul 10 20:17:53.149983 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:53.149948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8" event={"ID":"3806b8a5-3878-4896-9df1-070f3d9b6f3d","Type":"ContainerStarted","Data":"c234414d44113dac40d379a8547a21945ca4240a4e136094e65c2b0056fce763"}
Jul 10 20:17:53.168863 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:17:53.168808    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8" podStartSLOduration=1.731639414 podStartE2EDuration="3.168764011s" totalImagesPullingTime="1.437124597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:17:53.167874343 +0000 UTC m=+560.393641299" watchObservedRunningTime="2026-07-10 20:17:53.168764011 +0000 UTC m=+560.394530940"
Jul 10 20:18:00.846581 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:00.846536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:18:00.847000 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:00.846606    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:18:00.859300 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:00.859275    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:18:01.201304 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:01.201220    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8"
Jul 10 20:18:30.548302 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.548251    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"]
Jul 10 20:18:30.551169 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.551153    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.554088 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.554070    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 10 20:18:30.565329 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.565304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"]
Jul 10 20:18:30.638124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.638090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.638124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.638124    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ffeb654-0465-476c-ad30-c0f954c16e98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.638329 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.638167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.638329 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.638263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.638329 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.638290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2prg\" (UniqueName: \"kubernetes.io/projected/9ffeb654-0465-476c-ad30-c0f954c16e98-kube-api-access-n2prg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.638425 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.638335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.738983 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.738946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.738983 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.738983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2prg\" (UniqueName: \"kubernetes.io/projected/9ffeb654-0465-476c-ad30-c0f954c16e98-kube-api-access-n2prg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.739196 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.739025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.739196 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.739055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.739196 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.739078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ffeb654-0465-476c-ad30-c0f954c16e98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.739196 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.739142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.739413 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.739390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.739474 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.739408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.739474 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.739462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.741198 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.741176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ffeb654-0465-476c-ad30-c0f954c16e98-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.741525 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.741505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ffeb654-0465-476c-ad30-c0f954c16e98-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.746298 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.746276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2prg\" (UniqueName: \"kubernetes.io/projected/9ffeb654-0465-476c-ad30-c0f954c16e98-kube-api-access-n2prg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xhq5t\" (UID: \"9ffeb654-0465-476c-ad30-c0f954c16e98\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.861380 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.861282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:30.994002 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:30.993971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"]
Jul 10 20:18:30.995683 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:18:30.995648    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ffeb654_0465_476c_ad30_c0f954c16e98.slice/crio-05adee1e8bfc27ca27f5abf2de4b0a9e8f0dc273f5b2f743f17a8a07bb379800 WatchSource:0}: Error finding container 05adee1e8bfc27ca27f5abf2de4b0a9e8f0dc273f5b2f743f17a8a07bb379800: Status 404 returned error can't find the container with id 05adee1e8bfc27ca27f5abf2de4b0a9e8f0dc273f5b2f743f17a8a07bb379800
Jul 10 20:18:31.346485 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:31.346446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t" event={"ID":"9ffeb654-0465-476c-ad30-c0f954c16e98","Type":"ContainerStarted","Data":"7f6408060bf8cb6d20125ba2f552d938633fbd0900c971de262d1fe1a24e1aba"}
Jul 10 20:18:31.346714 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:31.346504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t" event={"ID":"9ffeb654-0465-476c-ad30-c0f954c16e98","Type":"ContainerStarted","Data":"05adee1e8bfc27ca27f5abf2de4b0a9e8f0dc273f5b2f743f17a8a07bb379800"}
Jul 10 20:18:31.365065 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:31.365014    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t" podStartSLOduration=1.184833878 podStartE2EDuration="1.36499947s" totalImagesPullingTime="180.165592ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:18:31.363570756 +0000 UTC m=+598.589337686" watchObservedRunningTime="2026-07-10 20:18:31.36499947 +0000 UTC m=+598.590766400"
Jul 10 20:18:33.172159 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:33.172130    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:18:33.175305 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:33.175283    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:18:40.861637 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:40.861608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:40.864241 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:40.861882    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:40.874300 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:40.874278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:18:41.408592 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:18:41.408565    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xhq5t"
Jul 10 20:20:49.717076 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:49.717041    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-sftdl"]
Jul 10 20:20:49.717621 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:49.717273    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56c7cffc4c-sftdl" podUID="9788e735-79a2-4719-bdb6-47bd280a3b48" containerName="manager" containerID="cri-o://49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72" gracePeriod=30
Jul 10 20:20:49.962078 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:49.962055    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:20:50.075013 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.074975    2580 generic.go:356] "Generic (PLEG): container finished" podID="9788e735-79a2-4719-bdb6-47bd280a3b48" containerID="49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72" exitCode=0
Jul 10 20:20:50.075185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.075025    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c7cffc4c-sftdl"
Jul 10 20:20:50.075185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.075062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c7cffc4c-sftdl" event={"ID":"9788e735-79a2-4719-bdb6-47bd280a3b48","Type":"ContainerDied","Data":"49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72"}
Jul 10 20:20:50.075185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.075116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c7cffc4c-sftdl" event={"ID":"9788e735-79a2-4719-bdb6-47bd280a3b48","Type":"ContainerDied","Data":"7066eb613c50e151206bf52b6b8ed4c51f89ab48c8e57e0ac23e149122ebdbd1"}
Jul 10 20:20:50.075185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.075130    2580 scope.go:117] "RemoveContainer" containerID="49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72"
Jul 10 20:20:50.081805 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.081780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d6sq\" (UniqueName: \"kubernetes.io/projected/9788e735-79a2-4719-bdb6-47bd280a3b48-kube-api-access-9d6sq\") pod \"9788e735-79a2-4719-bdb6-47bd280a3b48\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") "
Jul 10 20:20:50.081926 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.081847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert\") pod \"9788e735-79a2-4719-bdb6-47bd280a3b48\" (UID: \"9788e735-79a2-4719-bdb6-47bd280a3b48\") "
Jul 10 20:20:50.083836 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.083817    2580 scope.go:117] "RemoveContainer" containerID="49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72"
Jul 10 20:20:50.083966 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.083942    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9788e735-79a2-4719-bdb6-47bd280a3b48" (UID: "9788e735-79a2-4719-bdb6-47bd280a3b48"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:20:50.084045 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.083963    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9788e735-79a2-4719-bdb6-47bd280a3b48-kube-api-access-9d6sq" (OuterVolumeSpecName: "kube-api-access-9d6sq") pod "9788e735-79a2-4719-bdb6-47bd280a3b48" (UID: "9788e735-79a2-4719-bdb6-47bd280a3b48"). InnerVolumeSpecName "kube-api-access-9d6sq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:20:50.084119 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:20:50.084100    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72\": container with ID starting with 49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72 not found: ID does not exist" containerID="49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72"
Jul 10 20:20:50.084158 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.084126    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72"} err="failed to get container status \"49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72\": rpc error: code = NotFound desc = could not find container \"49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72\": container with ID starting with 49a7be8630d41022aaf9c675a3f6355a84907254b2dd1de3489818c58a869d72 not found: ID does not exist"
Jul 10 20:20:50.183028 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.182986    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d6sq\" (UniqueName: \"kubernetes.io/projected/9788e735-79a2-4719-bdb6-47bd280a3b48-kube-api-access-9d6sq\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:20:50.183028 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.183021    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9788e735-79a2-4719-bdb6-47bd280a3b48-webhook-cert\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:20:50.397766 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.397732    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-sftdl"]
Jul 10 20:20:50.401917 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:50.401893    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-sftdl"]
Jul 10 20:20:51.207138 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.207106    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-lcfjd"]
Jul 10 20:20:51.207507 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.207467    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9788e735-79a2-4719-bdb6-47bd280a3b48" containerName="manager"
Jul 10 20:20:51.207507 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.207482    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9788e735-79a2-4719-bdb6-47bd280a3b48" containerName="manager"
Jul 10 20:20:51.207596 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.207578    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9788e735-79a2-4719-bdb6-47bd280a3b48" containerName="manager"
Jul 10 20:20:51.211863 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.211847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.215750 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.215714    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vtb2p\""
Jul 10 20:20:51.215750 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.215742    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 20:20:51.215926 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.215742    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 20:20:51.221860 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.221837    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-lcfjd"]
Jul 10 20:20:51.281218 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.281179    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9788e735-79a2-4719-bdb6-47bd280a3b48" path="/var/lib/kubelet/pods/9788e735-79a2-4719-bdb6-47bd280a3b48/volumes"
Jul 10 20:20:51.293366 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.293343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr8rq\" (UniqueName: \"kubernetes.io/projected/1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4-kube-api-access-vr8rq\") pod \"maas-controller-56c7cffc4c-lcfjd\" (UID: \"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4\") " pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.293457 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.293405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4-webhook-cert\") pod \"maas-controller-56c7cffc4c-lcfjd\" (UID: \"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4\") " pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.394457 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.394426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr8rq\" (UniqueName: \"kubernetes.io/projected/1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4-kube-api-access-vr8rq\") pod \"maas-controller-56c7cffc4c-lcfjd\" (UID: \"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4\") " pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.394621 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.394485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4-webhook-cert\") pod \"maas-controller-56c7cffc4c-lcfjd\" (UID: \"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4\") " pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.396861 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.396834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4-webhook-cert\") pod \"maas-controller-56c7cffc4c-lcfjd\" (UID: \"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4\") " pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.402824 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.402800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr8rq\" (UniqueName: \"kubernetes.io/projected/1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4-kube-api-access-vr8rq\") pod \"maas-controller-56c7cffc4c-lcfjd\" (UID: \"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4\") " pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.522709 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.522659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:51.653821 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.653795    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c7cffc4c-lcfjd"]
Jul 10 20:20:51.655368 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:20:51.655336    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2bcd25_2978_4d64_bd73_a7e4b6f72ae4.slice/crio-259e18ae422074381a8a1adf211adb2fd4463916690e18a8b903b48272f6baf4 WatchSource:0}: Error finding container 259e18ae422074381a8a1adf211adb2fd4463916690e18a8b903b48272f6baf4: Status 404 returned error can't find the container with id 259e18ae422074381a8a1adf211adb2fd4463916690e18a8b903b48272f6baf4
Jul 10 20:20:51.656772 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:51.656753    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:20:52.089462 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:52.089436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c7cffc4c-lcfjd" event={"ID":"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4","Type":"ContainerStarted","Data":"259e18ae422074381a8a1adf211adb2fd4463916690e18a8b903b48272f6baf4"}
Jul 10 20:20:53.096155 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:53.096121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c7cffc4c-lcfjd" event={"ID":"1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4","Type":"ContainerStarted","Data":"6df0f042fe4d891cf0742f596a7857e654f907d50374c9add70a5371d2bf472f"}
Jul 10 20:20:53.096520 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:53.096223    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:20:53.115836 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:20:53.115789    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56c7cffc4c-lcfjd" podStartSLOduration=1.707668024 podStartE2EDuration="2.115773919s" totalImagesPullingTime="408.105895ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:20:53.112619068 +0000 UTC m=+740.338386000" watchObservedRunningTime="2026-07-10 20:20:53.115773919 +0000 UTC m=+740.341540848"
Jul 10 20:21:04.107274 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:21:04.107240    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56c7cffc4c-lcfjd"
Jul 10 20:23:33.206484 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:23:33.206417    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:23:33.206484 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:23:33.206473    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:28:33.239916 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:28:33.239888    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:28:33.241931 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:28:33.241913    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:33:00.589733 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.589694    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"]
Jul 10 20:33:00.593843 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.593799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.596383 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.596360    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-05f0a8c8-serving-cert\""
Jul 10 20:33:00.605324 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.605300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"]
Jul 10 20:33:00.662922 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.662879    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5416cc-d170-4891-b649-898d5ec4a5d8-openshift-service-ca\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.663106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.662942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mknr\" (UniqueName: \"kubernetes.io/projected/bb5416cc-d170-4891-b649-898d5ec4a5d8-kube-api-access-7mknr\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.663106 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.663000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.764466 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.764427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.764719 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.764495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5416cc-d170-4891-b649-898d5ec4a5d8-openshift-service-ca\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.764719 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.764529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mknr\" (UniqueName: \"kubernetes.io/projected/bb5416cc-d170-4891-b649-898d5ec4a5d8-kube-api-access-7mknr\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.764719 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:33:00.764602    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-serving-cert: secret "maas-api-e2e-ait-05f0a8c8-serving-cert" not found
Jul 10 20:33:00.764719 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:33:00.764713    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls podName:bb5416cc-d170-4891-b649-898d5ec4a5d8 nodeName:}" failed. No retries permitted until 2026-07-10 20:33:01.264661235 +0000 UTC m=+1468.490428143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls") pod "maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r" (UID: "bb5416cc-d170-4891-b649-898d5ec4a5d8") : secret "maas-api-e2e-ait-05f0a8c8-serving-cert" not found
Jul 10 20:33:00.765140 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.765122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5416cc-d170-4891-b649-898d5ec4a5d8-openshift-service-ca\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:00.774601 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:00.774571    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mknr\" (UniqueName: \"kubernetes.io/projected/bb5416cc-d170-4891-b649-898d5ec4a5d8-kube-api-access-7mknr\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:01.270604 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:01.270560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:01.273021 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:01.272991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls\") pod \"maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:01.505825 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:01.505787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:01.649410 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:01.649379    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"]
Jul 10 20:33:01.650881 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:33:01.650856    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5416cc_d170_4891_b649_898d5ec4a5d8.slice/crio-6ef4cad4526bc2a280030f46b7ec343153b1328d44c2a3e9057051a1a1596a5e WatchSource:0}: Error finding container 6ef4cad4526bc2a280030f46b7ec343153b1328d44c2a3e9057051a1a1596a5e: Status 404 returned error can't find the container with id 6ef4cad4526bc2a280030f46b7ec343153b1328d44c2a3e9057051a1a1596a5e
Jul 10 20:33:01.652226 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:01.652206    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:33:01.811784 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:01.811698    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r" event={"ID":"bb5416cc-d170-4891-b649-898d5ec4a5d8","Type":"ContainerStarted","Data":"6ef4cad4526bc2a280030f46b7ec343153b1328d44c2a3e9057051a1a1596a5e"}
Jul 10 20:33:02.818603 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:02.818567    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r" event={"ID":"bb5416cc-d170-4891-b649-898d5ec4a5d8","Type":"ContainerStarted","Data":"b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea"}
Jul 10 20:33:02.819039 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:02.818652    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:02.836742 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:02.836686    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r" podStartSLOduration=2.222102271 podStartE2EDuration="2.836652404s" totalImagesPullingTime="614.550133ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:33:02.834274159 +0000 UTC m=+1470.060041089" watchObservedRunningTime="2026-07-10 20:33:02.836652404 +0000 UTC m=+1470.062419361"
Jul 10 20:33:05.915033 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:05.915005    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"]
Jul 10 20:33:05.915461 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:05.915176    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r" podUID="bb5416cc-d170-4891-b649-898d5ec4a5d8" containerName="maas-api" containerID="cri-o://b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea" gracePeriod=30
Jul 10 20:33:06.167645 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.167582    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:06.218316 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.218290    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls\") pod \"bb5416cc-d170-4891-b649-898d5ec4a5d8\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") "
Jul 10 20:33:06.218473 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.218326    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mknr\" (UniqueName: \"kubernetes.io/projected/bb5416cc-d170-4891-b649-898d5ec4a5d8-kube-api-access-7mknr\") pod \"bb5416cc-d170-4891-b649-898d5ec4a5d8\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") "
Jul 10 20:33:06.218473 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.218381    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5416cc-d170-4891-b649-898d5ec4a5d8-openshift-service-ca\") pod \"bb5416cc-d170-4891-b649-898d5ec4a5d8\" (UID: \"bb5416cc-d170-4891-b649-898d5ec4a5d8\") "
Jul 10 20:33:06.218834 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.218810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5416cc-d170-4891-b649-898d5ec4a5d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb5416cc-d170-4891-b649-898d5ec4a5d8" (UID: "bb5416cc-d170-4891-b649-898d5ec4a5d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:33:06.220585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.220554    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb5416cc-d170-4891-b649-898d5ec4a5d8" (UID: "bb5416cc-d170-4891-b649-898d5ec4a5d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:33:06.220765 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.220604    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5416cc-d170-4891-b649-898d5ec4a5d8-kube-api-access-7mknr" (OuterVolumeSpecName: "kube-api-access-7mknr") pod "bb5416cc-d170-4891-b649-898d5ec4a5d8" (UID: "bb5416cc-d170-4891-b649-898d5ec4a5d8"). InnerVolumeSpecName "kube-api-access-7mknr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:33:06.319301 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.319267    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb5416cc-d170-4891-b649-898d5ec4a5d8-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:06.319301 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.319298    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mknr\" (UniqueName: \"kubernetes.io/projected/bb5416cc-d170-4891-b649-898d5ec4a5d8-kube-api-access-7mknr\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:06.319301 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.319310    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb5416cc-d170-4891-b649-898d5ec4a5d8-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:06.839560 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.839522    2580 generic.go:356] "Generic (PLEG): container finished" podID="bb5416cc-d170-4891-b649-898d5ec4a5d8" containerID="b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea" exitCode=0
Jul 10 20:33:06.839758 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.839573    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"
Jul 10 20:33:06.839758 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.839603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r" event={"ID":"bb5416cc-d170-4891-b649-898d5ec4a5d8","Type":"ContainerDied","Data":"b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea"}
Jul 10 20:33:06.839758 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.839657    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r" event={"ID":"bb5416cc-d170-4891-b649-898d5ec4a5d8","Type":"ContainerDied","Data":"6ef4cad4526bc2a280030f46b7ec343153b1328d44c2a3e9057051a1a1596a5e"}
Jul 10 20:33:06.839758 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.839692    2580 scope.go:117] "RemoveContainer" containerID="b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea"
Jul 10 20:33:06.848626 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.848601    2580 scope.go:117] "RemoveContainer" containerID="b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea"
Jul 10 20:33:06.848914 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:33:06.848895    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea\": container with ID starting with b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea not found: ID does not exist" containerID="b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea"
Jul 10 20:33:06.848971 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.848922    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea"} err="failed to get container status \"b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea\": rpc error: code = NotFound desc = could not find container \"b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea\": container with ID starting with b94aca23ae1bdc295ae553e1b8a875dc14818d7c375724c1b13817bdc03d54ea not found: ID does not exist"
Jul 10 20:33:06.860934 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.860908    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"]
Jul 10 20:33:06.863833 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:06.863806    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-05f0a8c8-57c9bdd957-pvk8r"]
Jul 10 20:33:07.281684 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:07.281637    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb5416cc-d170-4891-b649-898d5ec4a5d8" path="/var/lib/kubelet/pods/bb5416cc-d170-4891-b649-898d5ec4a5d8/volumes"
Jul 10 20:33:13.005717 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.005663    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"]
Jul 10 20:33:13.006208 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.006183    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5416cc-d170-4891-b649-898d5ec4a5d8" containerName="maas-api"
Jul 10 20:33:13.006208 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.006208    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5416cc-d170-4891-b649-898d5ec4a5d8" containerName="maas-api"
Jul 10 20:33:13.006410 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.006327    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5416cc-d170-4891-b649-898d5ec4a5d8" containerName="maas-api"
Jul 10 20:33:13.010781 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.010763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.013682 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.013650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-645f4009-serving-cert\""
Jul 10 20:33:13.024862 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.024841    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"]
Jul 10 20:33:13.078663 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.078628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33612ef1-e61d-4364-a860-0ac537493a4e-maas-api-tls\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.078861 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.078718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33612ef1-e61d-4364-a860-0ac537493a4e-openshift-service-ca\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.078861 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.078763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpzqj\" (UniqueName: \"kubernetes.io/projected/33612ef1-e61d-4364-a860-0ac537493a4e-kube-api-access-zpzqj\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.179916 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.179876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33612ef1-e61d-4364-a860-0ac537493a4e-maas-api-tls\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.180114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.179943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33612ef1-e61d-4364-a860-0ac537493a4e-openshift-service-ca\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.180114 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.179986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpzqj\" (UniqueName: \"kubernetes.io/projected/33612ef1-e61d-4364-a860-0ac537493a4e-kube-api-access-zpzqj\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.180696 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.180650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33612ef1-e61d-4364-a860-0ac537493a4e-openshift-service-ca\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.182297 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.182276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33612ef1-e61d-4364-a860-0ac537493a4e-maas-api-tls\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.190165 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.190145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpzqj\" (UniqueName: \"kubernetes.io/projected/33612ef1-e61d-4364-a860-0ac537493a4e-kube-api-access-zpzqj\") pod \"maas-api-e2e-ait-645f4009-848c8d85dd-8kbws\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.321236 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.321202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:13.463300 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.463270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"]
Jul 10 20:33:13.465926 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:33:13.465896    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33612ef1_e61d_4364_a860_0ac537493a4e.slice/crio-ee9348d8eb9d226f55ab84e2755d5beff02df2d1fdc518a316a48d5c2001b479 WatchSource:0}: Error finding container ee9348d8eb9d226f55ab84e2755d5beff02df2d1fdc518a316a48d5c2001b479: Status 404 returned error can't find the container with id ee9348d8eb9d226f55ab84e2755d5beff02df2d1fdc518a316a48d5c2001b479
Jul 10 20:33:13.879205 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:13.879172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws" event={"ID":"33612ef1-e61d-4364-a860-0ac537493a4e","Type":"ContainerStarted","Data":"ee9348d8eb9d226f55ab84e2755d5beff02df2d1fdc518a316a48d5c2001b479"}
Jul 10 20:33:14.885138 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:14.885105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws" event={"ID":"33612ef1-e61d-4364-a860-0ac537493a4e","Type":"ContainerStarted","Data":"7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5"}
Jul 10 20:33:14.885527 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:14.885237    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:14.903526 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:14.903472    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws" podStartSLOduration=2.439691482 podStartE2EDuration="2.903455265s" totalImagesPullingTime="463.763783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:33:14.900485215 +0000 UTC m=+1482.126252145" watchObservedRunningTime="2026-07-10 20:33:14.903455265 +0000 UTC m=+1482.129222193"
Jul 10 20:33:18.320331 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.320295    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"]
Jul 10 20:33:18.320745 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.320498    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws" podUID="33612ef1-e61d-4364-a860-0ac537493a4e" containerName="maas-api" containerID="cri-o://7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5" gracePeriod=30
Jul 10 20:33:18.325860 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.325836    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:18.576877 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.576821    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:18.728863 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.728828    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33612ef1-e61d-4364-a860-0ac537493a4e-openshift-service-ca\") pod \"33612ef1-e61d-4364-a860-0ac537493a4e\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") "
Jul 10 20:33:18.729051 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.728875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33612ef1-e61d-4364-a860-0ac537493a4e-maas-api-tls\") pod \"33612ef1-e61d-4364-a860-0ac537493a4e\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") "
Jul 10 20:33:18.729051 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.728933    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpzqj\" (UniqueName: \"kubernetes.io/projected/33612ef1-e61d-4364-a860-0ac537493a4e-kube-api-access-zpzqj\") pod \"33612ef1-e61d-4364-a860-0ac537493a4e\" (UID: \"33612ef1-e61d-4364-a860-0ac537493a4e\") "
Jul 10 20:33:18.729500 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.729467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33612ef1-e61d-4364-a860-0ac537493a4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33612ef1-e61d-4364-a860-0ac537493a4e" (UID: "33612ef1-e61d-4364-a860-0ac537493a4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:33:18.731142 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.731096    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33612ef1-e61d-4364-a860-0ac537493a4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33612ef1-e61d-4364-a860-0ac537493a4e" (UID: "33612ef1-e61d-4364-a860-0ac537493a4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:33:18.731289 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.731185    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33612ef1-e61d-4364-a860-0ac537493a4e-kube-api-access-zpzqj" (OuterVolumeSpecName: "kube-api-access-zpzqj") pod "33612ef1-e61d-4364-a860-0ac537493a4e" (UID: "33612ef1-e61d-4364-a860-0ac537493a4e"). InnerVolumeSpecName "kube-api-access-zpzqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:33:18.829890 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.829804    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33612ef1-e61d-4364-a860-0ac537493a4e-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:18.829890 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.829836    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33612ef1-e61d-4364-a860-0ac537493a4e-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:18.829890 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.829847    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpzqj\" (UniqueName: \"kubernetes.io/projected/33612ef1-e61d-4364-a860-0ac537493a4e-kube-api-access-zpzqj\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:18.906063 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.906030    2580 generic.go:356] "Generic (PLEG): container finished" podID="33612ef1-e61d-4364-a860-0ac537493a4e" containerID="7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5" exitCode=0
Jul 10 20:33:18.906257 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.906080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"
Jul 10 20:33:18.906257 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.906116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws" event={"ID":"33612ef1-e61d-4364-a860-0ac537493a4e","Type":"ContainerDied","Data":"7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5"}
Jul 10 20:33:18.906257 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.906179    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws" event={"ID":"33612ef1-e61d-4364-a860-0ac537493a4e","Type":"ContainerDied","Data":"ee9348d8eb9d226f55ab84e2755d5beff02df2d1fdc518a316a48d5c2001b479"}
Jul 10 20:33:18.906257 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.906210    2580 scope.go:117] "RemoveContainer" containerID="7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5"
Jul 10 20:33:18.914996 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.914980    2580 scope.go:117] "RemoveContainer" containerID="7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5"
Jul 10 20:33:18.915266 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:33:18.915248    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5\": container with ID starting with 7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5 not found: ID does not exist" containerID="7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5"
Jul 10 20:33:18.915337 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.915275    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5"} err="failed to get container status \"7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5\": rpc error: code = NotFound desc = could not find container \"7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5\": container with ID starting with 7d78260f8caab3e8731af2085eb65c789d8b400f6a2b6b25ef2258f79a70a6f5 not found: ID does not exist"
Jul 10 20:33:18.930796 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.930765    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"]
Jul 10 20:33:18.934572 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:18.934548    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-645f4009-848c8d85dd-8kbws"]
Jul 10 20:33:19.281862 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:19.281824    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33612ef1-e61d-4364-a860-0ac537493a4e" path="/var/lib/kubelet/pods/33612ef1-e61d-4364-a860-0ac537493a4e/volumes"
Jul 10 20:33:31.458584 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.458548    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"]
Jul 10 20:33:31.459111 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.459096    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33612ef1-e61d-4364-a860-0ac537493a4e" containerName="maas-api"
Jul 10 20:33:31.459160 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.459115    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33612ef1-e61d-4364-a860-0ac537493a4e" containerName="maas-api"
Jul 10 20:33:31.459216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.459205    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33612ef1-e61d-4364-a860-0ac537493a4e" containerName="maas-api"
Jul 10 20:33:31.464085 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.464067    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.466610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.466590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-cc2a2b5f-serving-cert\""
Jul 10 20:33:31.471944 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.471920    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"]
Jul 10 20:33:31.543849 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.543805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaaa2bdc-4318-4590-886f-87ba2983018b-maas-api-tls\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.544035 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.543932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaaa2bdc-4318-4590-886f-87ba2983018b-openshift-service-ca\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.544035 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.543983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnl6\" (UniqueName: \"kubernetes.io/projected/eaaa2bdc-4318-4590-886f-87ba2983018b-kube-api-access-fmnl6\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.645224 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.645196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnl6\" (UniqueName: \"kubernetes.io/projected/eaaa2bdc-4318-4590-886f-87ba2983018b-kube-api-access-fmnl6\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.645385 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.645284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaaa2bdc-4318-4590-886f-87ba2983018b-maas-api-tls\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.645385 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.645322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaaa2bdc-4318-4590-886f-87ba2983018b-openshift-service-ca\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.645918 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.645897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaaa2bdc-4318-4590-886f-87ba2983018b-openshift-service-ca\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.647762 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.647739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaaa2bdc-4318-4590-886f-87ba2983018b-maas-api-tls\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.656292 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.656266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnl6\" (UniqueName: \"kubernetes.io/projected/eaaa2bdc-4318-4590-886f-87ba2983018b-kube-api-access-fmnl6\") pod \"maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.775806 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.775780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:31.913097 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.913069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"]
Jul 10 20:33:31.914208 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:33:31.914183    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaaa2bdc_4318_4590_886f_87ba2983018b.slice/crio-d668c2432b7493d57ae2bfd4025d142e2b05adf987ede6ff3d21fd7014a6263d WatchSource:0}: Error finding container d668c2432b7493d57ae2bfd4025d142e2b05adf987ede6ff3d21fd7014a6263d: Status 404 returned error can't find the container with id d668c2432b7493d57ae2bfd4025d142e2b05adf987ede6ff3d21fd7014a6263d
Jul 10 20:33:31.975935 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:31.975899    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8" event={"ID":"eaaa2bdc-4318-4590-886f-87ba2983018b","Type":"ContainerStarted","Data":"d668c2432b7493d57ae2bfd4025d142e2b05adf987ede6ff3d21fd7014a6263d"}
Jul 10 20:33:32.982322 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:32.982285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8" event={"ID":"eaaa2bdc-4318-4590-886f-87ba2983018b","Type":"ContainerStarted","Data":"02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14"}
Jul 10 20:33:32.982789 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:32.982378    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:32.999908 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:32.999860    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8" podStartSLOduration=1.5916900489999999 podStartE2EDuration="1.99984462s" totalImagesPullingTime="408.154571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:33:32.998124387 +0000 UTC m=+1500.223891329" watchObservedRunningTime="2026-07-10 20:33:32.99984462 +0000 UTC m=+1500.225611549"
Jul 10 20:33:33.279049 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:33.279017    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:33:33.281635 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:33.281609    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:33:36.276496 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.276458    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"]
Jul 10 20:33:36.276937 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.276642    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8" podUID="eaaa2bdc-4318-4590-886f-87ba2983018b" containerName="maas-api" containerID="cri-o://02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14" gracePeriod=30
Jul 10 20:33:36.524871 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.524847    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:36.691730 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.691645    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmnl6\" (UniqueName: \"kubernetes.io/projected/eaaa2bdc-4318-4590-886f-87ba2983018b-kube-api-access-fmnl6\") pod \"eaaa2bdc-4318-4590-886f-87ba2983018b\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") "
Jul 10 20:33:36.691730 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.691709    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaaa2bdc-4318-4590-886f-87ba2983018b-maas-api-tls\") pod \"eaaa2bdc-4318-4590-886f-87ba2983018b\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") "
Jul 10 20:33:36.691914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.691750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaaa2bdc-4318-4590-886f-87ba2983018b-openshift-service-ca\") pod \"eaaa2bdc-4318-4590-886f-87ba2983018b\" (UID: \"eaaa2bdc-4318-4590-886f-87ba2983018b\") "
Jul 10 20:33:36.692144 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.692111    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaaa2bdc-4318-4590-886f-87ba2983018b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eaaa2bdc-4318-4590-886f-87ba2983018b" (UID: "eaaa2bdc-4318-4590-886f-87ba2983018b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:33:36.693873 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.693844    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaaa2bdc-4318-4590-886f-87ba2983018b-kube-api-access-fmnl6" (OuterVolumeSpecName: "kube-api-access-fmnl6") pod "eaaa2bdc-4318-4590-886f-87ba2983018b" (UID: "eaaa2bdc-4318-4590-886f-87ba2983018b"). InnerVolumeSpecName "kube-api-access-fmnl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:33:36.693967 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.693932    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaaa2bdc-4318-4590-886f-87ba2983018b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eaaa2bdc-4318-4590-886f-87ba2983018b" (UID: "eaaa2bdc-4318-4590-886f-87ba2983018b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:33:36.792909 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.792872    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmnl6\" (UniqueName: \"kubernetes.io/projected/eaaa2bdc-4318-4590-886f-87ba2983018b-kube-api-access-fmnl6\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:36.792909 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.792904    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaaa2bdc-4318-4590-886f-87ba2983018b-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:36.792909 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:36.792913    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaaa2bdc-4318-4590-886f-87ba2983018b-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:33:37.006458 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.006371    2580 generic.go:356] "Generic (PLEG): container finished" podID="eaaa2bdc-4318-4590-886f-87ba2983018b" containerID="02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14" exitCode=0
Jul 10 20:33:37.006458 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.006434    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"
Jul 10 20:33:37.006629 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.006466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8" event={"ID":"eaaa2bdc-4318-4590-886f-87ba2983018b","Type":"ContainerDied","Data":"02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14"}
Jul 10 20:33:37.006629 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.006520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8" event={"ID":"eaaa2bdc-4318-4590-886f-87ba2983018b","Type":"ContainerDied","Data":"d668c2432b7493d57ae2bfd4025d142e2b05adf987ede6ff3d21fd7014a6263d"}
Jul 10 20:33:37.006629 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.006542    2580 scope.go:117] "RemoveContainer" containerID="02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14"
Jul 10 20:33:37.020337 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.020296    2580 scope.go:117] "RemoveContainer" containerID="02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14"
Jul 10 20:33:37.020856 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:33:37.020830    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14\": container with ID starting with 02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14 not found: ID does not exist" containerID="02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14"
Jul 10 20:33:37.020949 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.020862    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14"} err="failed to get container status \"02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14\": rpc error: code = NotFound desc = could not find container \"02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14\": container with ID starting with 02224c1e477403f0610f3080c9dad7167bab639bb0c6f7a1977eecc3fcd52a14 not found: ID does not exist"
Jul 10 20:33:37.034538 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.034514    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"]
Jul 10 20:33:37.038648 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.038626    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-cc2a2b5f-5bb4b98b75-f6ch8"]
Jul 10 20:33:37.280961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:33:37.280928    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaaa2bdc-4318-4590-886f-87ba2983018b" path="/var/lib/kubelet/pods/eaaa2bdc-4318-4590-886f-87ba2983018b/volumes"
Jul 10 20:35:18.911854 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:18.911825    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"]
Jul 10 20:35:18.912319 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:18.912184    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaaa2bdc-4318-4590-886f-87ba2983018b" containerName="maas-api"
Jul 10 20:35:18.912319 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:18.912203    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaaa2bdc-4318-4590-886f-87ba2983018b" containerName="maas-api"
Jul 10 20:35:18.912319 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:18.912313    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaaa2bdc-4318-4590-886f-87ba2983018b" containerName="maas-api"
Jul 10 20:35:18.915483 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:18.915461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:18.918204 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:18.918178    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d291e727-serving-cert\""
Jul 10 20:35:18.926518 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:18.926492    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"]
Jul 10 20:35:19.002009 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.001970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45b24dce-5311-4189-a535-a91f21d4c9d3-maas-api-tls\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.002009 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.002010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45b24dce-5311-4189-a535-a91f21d4c9d3-openshift-service-ca\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.002221 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.002035    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2v2v\" (UniqueName: \"kubernetes.io/projected/45b24dce-5311-4189-a535-a91f21d4c9d3-kube-api-access-j2v2v\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.103128 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.103097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45b24dce-5311-4189-a535-a91f21d4c9d3-maas-api-tls\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.103286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.103137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45b24dce-5311-4189-a535-a91f21d4c9d3-openshift-service-ca\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.103286 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.103175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2v2v\" (UniqueName: \"kubernetes.io/projected/45b24dce-5311-4189-a535-a91f21d4c9d3-kube-api-access-j2v2v\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.103861 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.103834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45b24dce-5311-4189-a535-a91f21d4c9d3-openshift-service-ca\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.105528 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.105494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45b24dce-5311-4189-a535-a91f21d4c9d3-maas-api-tls\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.112402 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.112378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2v2v\" (UniqueName: \"kubernetes.io/projected/45b24dce-5311-4189-a535-a91f21d4c9d3-kube-api-access-j2v2v\") pod \"maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.226829 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.226743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:19.372442 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.372418    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"]
Jul 10 20:35:19.373950 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:35:19.373925    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b24dce_5311_4189_a535_a91f21d4c9d3.slice/crio-d4b4a9146dd1ef2cc05a15c56da824f9048d9c177ebcba5cdd3b75ee96f5f398 WatchSource:0}: Error finding container d4b4a9146dd1ef2cc05a15c56da824f9048d9c177ebcba5cdd3b75ee96f5f398: Status 404 returned error can't find the container with id d4b4a9146dd1ef2cc05a15c56da824f9048d9c177ebcba5cdd3b75ee96f5f398
Jul 10 20:35:19.533739 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:19.533704    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd" event={"ID":"45b24dce-5311-4189-a535-a91f21d4c9d3","Type":"ContainerStarted","Data":"d4b4a9146dd1ef2cc05a15c56da824f9048d9c177ebcba5cdd3b75ee96f5f398"}
Jul 10 20:35:20.542842 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:20.542754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd" event={"ID":"45b24dce-5311-4189-a535-a91f21d4c9d3","Type":"ContainerStarted","Data":"90f68d920755987b05aef72253ed0401a9ff7c5e8d5319559ebaae0dbb845e2a"}
Jul 10 20:35:20.543251 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:20.542904    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:20.561108 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:20.561056    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd" podStartSLOduration=1.6455690729999999 podStartE2EDuration="2.561040684s" totalImagesPullingTime="915.471611ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:35:20.558224492 +0000 UTC m=+1607.783991421" watchObservedRunningTime="2026-07-10 20:35:20.561040684 +0000 UTC m=+1607.786807613"
Jul 10 20:35:26.554571 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:26.554535    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:31.431403 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.431361    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"]
Jul 10 20:35:31.431914 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.431616    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd" podUID="45b24dce-5311-4189-a535-a91f21d4c9d3" containerName="maas-api" containerID="cri-o://90f68d920755987b05aef72253ed0401a9ff7c5e8d5319559ebaae0dbb845e2a" gracePeriod=30
Jul 10 20:35:31.549863 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.549829    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd" podUID="45b24dce-5311-4189-a535-a91f21d4c9d3" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.43:8443/health\": dial tcp 10.133.0.43:8443: connect: connection refused"
Jul 10 20:35:31.604385 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.604357    2580 generic.go:356] "Generic (PLEG): container finished" podID="45b24dce-5311-4189-a535-a91f21d4c9d3" containerID="90f68d920755987b05aef72253ed0401a9ff7c5e8d5319559ebaae0dbb845e2a" exitCode=0
Jul 10 20:35:31.604523 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.604430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd" event={"ID":"45b24dce-5311-4189-a535-a91f21d4c9d3","Type":"ContainerDied","Data":"90f68d920755987b05aef72253ed0401a9ff7c5e8d5319559ebaae0dbb845e2a"}
Jul 10 20:35:31.680010 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.679985    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:31.717034 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.716952    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45b24dce-5311-4189-a535-a91f21d4c9d3-maas-api-tls\") pod \"45b24dce-5311-4189-a535-a91f21d4c9d3\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") "
Jul 10 20:35:31.717034 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.717020    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2v2v\" (UniqueName: \"kubernetes.io/projected/45b24dce-5311-4189-a535-a91f21d4c9d3-kube-api-access-j2v2v\") pod \"45b24dce-5311-4189-a535-a91f21d4c9d3\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") "
Jul 10 20:35:31.717230 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.717076    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45b24dce-5311-4189-a535-a91f21d4c9d3-openshift-service-ca\") pod \"45b24dce-5311-4189-a535-a91f21d4c9d3\" (UID: \"45b24dce-5311-4189-a535-a91f21d4c9d3\") "
Jul 10 20:35:31.717531 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.717498    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45b24dce-5311-4189-a535-a91f21d4c9d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45b24dce-5311-4189-a535-a91f21d4c9d3" (UID: "45b24dce-5311-4189-a535-a91f21d4c9d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:35:31.719039 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.719015    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45b24dce-5311-4189-a535-a91f21d4c9d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45b24dce-5311-4189-a535-a91f21d4c9d3" (UID: "45b24dce-5311-4189-a535-a91f21d4c9d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:35:31.719161 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.719147    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b24dce-5311-4189-a535-a91f21d4c9d3-kube-api-access-j2v2v" (OuterVolumeSpecName: "kube-api-access-j2v2v") pod "45b24dce-5311-4189-a535-a91f21d4c9d3" (UID: "45b24dce-5311-4189-a535-a91f21d4c9d3"). InnerVolumeSpecName "kube-api-access-j2v2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:35:31.817799 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.817770    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45b24dce-5311-4189-a535-a91f21d4c9d3-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:35:31.817799 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.817797    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2v2v\" (UniqueName: \"kubernetes.io/projected/45b24dce-5311-4189-a535-a91f21d4c9d3-kube-api-access-j2v2v\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:35:31.817994 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:31.817811    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45b24dce-5311-4189-a535-a91f21d4c9d3-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:35:32.610262 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:32.610236    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"
Jul 10 20:35:32.610664 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:32.610276    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd" event={"ID":"45b24dce-5311-4189-a535-a91f21d4c9d3","Type":"ContainerDied","Data":"d4b4a9146dd1ef2cc05a15c56da824f9048d9c177ebcba5cdd3b75ee96f5f398"}
Jul 10 20:35:32.610664 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:32.610340    2580 scope.go:117] "RemoveContainer" containerID="90f68d920755987b05aef72253ed0401a9ff7c5e8d5319559ebaae0dbb845e2a"
Jul 10 20:35:32.632258 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:32.632232    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"]
Jul 10 20:35:32.636197 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:32.636172    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d291e727-8584c8b7db-jpwqd"]
Jul 10 20:35:33.282490 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:33.282461    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45b24dce-5311-4189-a535-a91f21d4c9d3" path="/var/lib/kubelet/pods/45b24dce-5311-4189-a535-a91f21d4c9d3/volumes"
Jul 10 20:35:59.914641 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:59.914604    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"]
Jul 10 20:35:59.915124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:59.914996    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45b24dce-5311-4189-a535-a91f21d4c9d3" containerName="maas-api"
Jul 10 20:35:59.915124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:59.915008    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="45b24dce-5311-4189-a535-a91f21d4c9d3" containerName="maas-api"
Jul 10 20:35:59.915124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:59.915077    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="45b24dce-5311-4189-a535-a91f21d4c9d3" containerName="maas-api"
Jul 10 20:35:59.918096 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:59.918080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:35:59.920727 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:59.920707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-02c11db1-serving-cert\""
Jul 10 20:35:59.927783 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:35:59.927755    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"]
Jul 10 20:36:00.067585 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.067546    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmv6g\" (UniqueName: \"kubernetes.io/projected/33ae5614-d68f-48c7-8938-e565497c558b-kube-api-access-hmv6g\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.067819 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.067616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.067819 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.067649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ae5614-d68f-48c7-8938-e565497c558b-openshift-service-ca\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.168367 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.168278    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.168367 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.168318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ae5614-d68f-48c7-8938-e565497c558b-openshift-service-ca\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.168624 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.168383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmv6g\" (UniqueName: \"kubernetes.io/projected/33ae5614-d68f-48c7-8938-e565497c558b-kube-api-access-hmv6g\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.168624 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:36:00.168447    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-serving-cert: secret "maas-api-e2e-mt-02c11db1-serving-cert" not found
Jul 10 20:36:00.168624 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:36:00.168523    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls podName:33ae5614-d68f-48c7-8938-e565497c558b nodeName:}" failed. No retries permitted until 2026-07-10 20:36:00.668507075 +0000 UTC m=+1647.894273983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls") pod "maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb" (UID: "33ae5614-d68f-48c7-8938-e565497c558b") : secret "maas-api-e2e-mt-02c11db1-serving-cert" not found
Jul 10 20:36:00.169095 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.169071    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ae5614-d68f-48c7-8938-e565497c558b-openshift-service-ca\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.184165 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.184128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmv6g\" (UniqueName: \"kubernetes.io/projected/33ae5614-d68f-48c7-8938-e565497c558b-kube-api-access-hmv6g\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.674188 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.674157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.676529 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.676504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls\") pod \"maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.829724 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.829680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:00.971344 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:00.971314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"]
Jul 10 20:36:00.973280 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:36:00.973251    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ae5614_d68f_48c7_8938_e565497c558b.slice/crio-c5a9f8d0fc4df33f0d987d7d562032b069fd94bda6c9f647a6f8a23536d4fcc9 WatchSource:0}: Error finding container c5a9f8d0fc4df33f0d987d7d562032b069fd94bda6c9f647a6f8a23536d4fcc9: Status 404 returned error can't find the container with id c5a9f8d0fc4df33f0d987d7d562032b069fd94bda6c9f647a6f8a23536d4fcc9
Jul 10 20:36:01.773749 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:01.773709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb" event={"ID":"33ae5614-d68f-48c7-8938-e565497c558b","Type":"ContainerStarted","Data":"736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8"}
Jul 10 20:36:01.773948 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:01.773776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb" event={"ID":"33ae5614-d68f-48c7-8938-e565497c558b","Type":"ContainerStarted","Data":"c5a9f8d0fc4df33f0d987d7d562032b069fd94bda6c9f647a6f8a23536d4fcc9"}
Jul 10 20:36:01.773948 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:01.773804    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:01.802356 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:01.802306    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb" podStartSLOduration=2.358787906 podStartE2EDuration="2.802290348s" totalImagesPullingTime="443.502442ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:36:01.802267955 +0000 UTC m=+1649.028034883" watchObservedRunningTime="2026-07-10 20:36:01.802290348 +0000 UTC m=+1649.028057292"
Jul 10 20:36:07.784767 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:07.784733    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:31.991086 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:31.991049    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"]
Jul 10 20:36:31.991594 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:31.991327    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb" podUID="33ae5614-d68f-48c7-8938-e565497c558b" containerName="maas-api" containerID="cri-o://736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8" gracePeriod=30
Jul 10 20:36:32.238972 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.238946    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:32.362466 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.362428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmv6g\" (UniqueName: \"kubernetes.io/projected/33ae5614-d68f-48c7-8938-e565497c558b-kube-api-access-hmv6g\") pod \"33ae5614-d68f-48c7-8938-e565497c558b\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") "
Jul 10 20:36:32.362642 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.362486    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls\") pod \"33ae5614-d68f-48c7-8938-e565497c558b\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") "
Jul 10 20:36:32.362642 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.362518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ae5614-d68f-48c7-8938-e565497c558b-openshift-service-ca\") pod \"33ae5614-d68f-48c7-8938-e565497c558b\" (UID: \"33ae5614-d68f-48c7-8938-e565497c558b\") "
Jul 10 20:36:32.363189 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.362996    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33ae5614-d68f-48c7-8938-e565497c558b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33ae5614-d68f-48c7-8938-e565497c558b" (UID: "33ae5614-d68f-48c7-8938-e565497c558b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:36:32.363332 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.363228    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33ae5614-d68f-48c7-8938-e565497c558b-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:36:32.365195 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.365171    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33ae5614-d68f-48c7-8938-e565497c558b" (UID: "33ae5614-d68f-48c7-8938-e565497c558b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:36:32.365195 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.365175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ae5614-d68f-48c7-8938-e565497c558b-kube-api-access-hmv6g" (OuterVolumeSpecName: "kube-api-access-hmv6g") pod "33ae5614-d68f-48c7-8938-e565497c558b" (UID: "33ae5614-d68f-48c7-8938-e565497c558b"). InnerVolumeSpecName "kube-api-access-hmv6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:36:32.464778 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.464739    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmv6g\" (UniqueName: \"kubernetes.io/projected/33ae5614-d68f-48c7-8938-e565497c558b-kube-api-access-hmv6g\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:36:32.464778 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.464772    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33ae5614-d68f-48c7-8938-e565497c558b-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:36:32.930255 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.930223    2580 generic.go:356] "Generic (PLEG): container finished" podID="33ae5614-d68f-48c7-8938-e565497c558b" containerID="736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8" exitCode=0
Jul 10 20:36:32.930422 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.930286    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"
Jul 10 20:36:32.930422 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.930307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb" event={"ID":"33ae5614-d68f-48c7-8938-e565497c558b","Type":"ContainerDied","Data":"736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8"}
Jul 10 20:36:32.930422 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.930357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb" event={"ID":"33ae5614-d68f-48c7-8938-e565497c558b","Type":"ContainerDied","Data":"c5a9f8d0fc4df33f0d987d7d562032b069fd94bda6c9f647a6f8a23536d4fcc9"}
Jul 10 20:36:32.930422 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.930380    2580 scope.go:117] "RemoveContainer" containerID="736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8"
Jul 10 20:36:32.939862 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.939810    2580 scope.go:117] "RemoveContainer" containerID="736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8"
Jul 10 20:36:32.940095 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:36:32.940075    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8\": container with ID starting with 736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8 not found: ID does not exist" containerID="736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8"
Jul 10 20:36:32.940159 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.940103    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8"} err="failed to get container status \"736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8\": rpc error: code = NotFound desc = could not find container \"736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8\": container with ID starting with 736e5b7df7b369ceb7e87308204f8891086d960cb5f7b626be7a098f3716a0e8 not found: ID does not exist"
Jul 10 20:36:32.954130 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.954108    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"]
Jul 10 20:36:32.960401 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:32.960381    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-02c11db1-6bfc766ddc-jcpnb"]
Jul 10 20:36:33.282756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:36:33.282729    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33ae5614-d68f-48c7-8938-e565497c558b" path="/var/lib/kubelet/pods/33ae5614-d68f-48c7-8938-e565497c558b/volumes"
Jul 10 20:38:29.802157 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.802121    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"]
Jul 10 20:38:29.802659 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.802644    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ae5614-d68f-48c7-8938-e565497c558b" containerName="maas-api"
Jul 10 20:38:29.802723 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.802662    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ae5614-d68f-48c7-8938-e565497c558b" containerName="maas-api"
Jul 10 20:38:29.802815 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.802803    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ae5614-d68f-48c7-8938-e565497c558b" containerName="maas-api"
Jul 10 20:38:29.806158 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.806139    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:29.809227 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.809206    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-58c4c1-serving-cert\""
Jul 10 20:38:29.822647 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.822583    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"]
Jul 10 20:38:29.892426 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.892386    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb42094-3bb5-4dd8-8884-9547b3a252b6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:29.892598 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.892432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:29.892598 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.892571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzbww\" (UniqueName: \"kubernetes.io/projected/1cb42094-3bb5-4dd8-8884-9547b3a252b6-kube-api-access-dzbww\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:29.993297 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.993252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzbww\" (UniqueName: \"kubernetes.io/projected/1cb42094-3bb5-4dd8-8884-9547b3a252b6-kube-api-access-dzbww\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:29.993470 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.993317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb42094-3bb5-4dd8-8884-9547b3a252b6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:29.993470 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.993341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:29.993544 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:38:29.993488    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-serving-cert: secret "maas-api-e2e-inf-a-58c4c1-serving-cert" not found
Jul 10 20:38:29.993597 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:38:29.993582    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls podName:1cb42094-3bb5-4dd8-8884-9547b3a252b6 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:30.493558128 +0000 UTC m=+1797.719325037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls") pod "maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp" (UID: "1cb42094-3bb5-4dd8-8884-9547b3a252b6") : secret "maas-api-e2e-inf-a-58c4c1-serving-cert" not found
Jul 10 20:38:29.994059 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:29.994040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb42094-3bb5-4dd8-8884-9547b3a252b6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:30.002374 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:30.002349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzbww\" (UniqueName: \"kubernetes.io/projected/1cb42094-3bb5-4dd8-8884-9547b3a252b6-kube-api-access-dzbww\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:30.496692 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:30.496630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:30.499050 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:30.499026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls\") pod \"maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:30.717848 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:30.717815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:30.880766 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:30.880742    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"]
Jul 10 20:38:30.882562 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:38:30.882536    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb42094_3bb5_4dd8_8884_9547b3a252b6.slice/crio-271293bbb6b5fce1611d537d626850808ca7a259551f44d4dd05c0ad47206697 WatchSource:0}: Error finding container 271293bbb6b5fce1611d537d626850808ca7a259551f44d4dd05c0ad47206697: Status 404 returned error can't find the container with id 271293bbb6b5fce1611d537d626850808ca7a259551f44d4dd05c0ad47206697
Jul 10 20:38:30.884054 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:30.884034    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 20:38:31.534320 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:31.534284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp" event={"ID":"1cb42094-3bb5-4dd8-8884-9547b3a252b6","Type":"ContainerStarted","Data":"9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96"}
Jul 10 20:38:31.534492 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:31.534341    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp" event={"ID":"1cb42094-3bb5-4dd8-8884-9547b3a252b6","Type":"ContainerStarted","Data":"271293bbb6b5fce1611d537d626850808ca7a259551f44d4dd05c0ad47206697"}
Jul 10 20:38:31.534492 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:31.534367    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:31.554844 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:31.554781    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp" podStartSLOduration=2.059237507 podStartE2EDuration="2.554759379s" totalImagesPullingTime="495.521872ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:31.551286622 +0000 UTC m=+1798.777053551" watchObservedRunningTime="2026-07-10 20:38:31.554759379 +0000 UTC m=+1798.780526312"
Jul 10 20:38:33.312715 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:33.312690    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:38:33.316864 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:33.316846    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:38:37.545161 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.545132    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:38:37.732566 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.732528    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"]
Jul 10 20:38:37.736083 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.736062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.739385 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.739366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-fba69b-serving-cert\""
Jul 10 20:38:37.750341 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.750315    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"]
Jul 10 20:38:37.867124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.867038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.867124 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.867077    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f216d124-c412-4e60-8df5-7ce224e00941-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.867364 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.867221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwjf7\" (UniqueName: \"kubernetes.io/projected/f216d124-c412-4e60-8df5-7ce224e00941-kube-api-access-wwjf7\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.968005 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.967964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f216d124-c412-4e60-8df5-7ce224e00941-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.968177 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.968072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwjf7\" (UniqueName: \"kubernetes.io/projected/f216d124-c412-4e60-8df5-7ce224e00941-kube-api-access-wwjf7\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.968177 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.968118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.968299 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:38:37.968230    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-serving-cert: secret "maas-api-e2e-inf-b-fba69b-serving-cert" not found
Jul 10 20:38:37.968341 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:38:37.968299    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls podName:f216d124-c412-4e60-8df5-7ce224e00941 nodeName:}" failed. No retries permitted until 2026-07-10 20:38:38.468279661 +0000 UTC m=+1805.694046577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls") pod "maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r" (UID: "f216d124-c412-4e60-8df5-7ce224e00941") : secret "maas-api-e2e-inf-b-fba69b-serving-cert" not found
Jul 10 20:38:37.968665 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.968644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f216d124-c412-4e60-8df5-7ce224e00941-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:37.978520 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:37.978496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwjf7\" (UniqueName: \"kubernetes.io/projected/f216d124-c412-4e60-8df5-7ce224e00941-kube-api-access-wwjf7\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:38.472899 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:38.472858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:38.475559 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:38.475526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls\") pod \"maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:38.647241 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:38.647202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:38.794971 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:38.794943    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"]
Jul 10 20:38:38.796618 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:38:38.796587    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf216d124_c412_4e60_8df5_7ce224e00941.slice/crio-89f5bf0c092832905fa40e95b6c2fdd9664d69fc20a6acf0c4a078e4f5af6fd1 WatchSource:0}: Error finding container 89f5bf0c092832905fa40e95b6c2fdd9664d69fc20a6acf0c4a078e4f5af6fd1: Status 404 returned error can't find the container with id 89f5bf0c092832905fa40e95b6c2fdd9664d69fc20a6acf0c4a078e4f5af6fd1
Jul 10 20:38:39.580425 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:39.580389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r" event={"ID":"f216d124-c412-4e60-8df5-7ce224e00941","Type":"ContainerStarted","Data":"726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a"}
Jul 10 20:38:39.580634 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:39.580448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r" event={"ID":"f216d124-c412-4e60-8df5-7ce224e00941","Type":"ContainerStarted","Data":"89f5bf0c092832905fa40e95b6c2fdd9664d69fc20a6acf0c4a078e4f5af6fd1"}
Jul 10 20:38:39.580634 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:39.580501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:38:39.598620 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:39.598564    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r" podStartSLOduration=2.331548992 podStartE2EDuration="2.598549249s" totalImagesPullingTime="267.000257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:38:39.59691257 +0000 UTC m=+1806.822679509" watchObservedRunningTime="2026-07-10 20:38:39.598549249 +0000 UTC m=+1806.824316178"
Jul 10 20:38:45.591543 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:38:45.591515    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:39:09.471860 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.471828    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"]
Jul 10 20:39:09.475313 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.475283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.480840 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.480815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fba69b\"/\"default-dockercfg-csknx\""
Jul 10 20:39:09.480974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.480815    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fba69b\"/\"kube-root-ca.crt\""
Jul 10 20:39:09.480974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.480915    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fba69b\"/\"openshift-service-ca.crt\""
Jul 10 20:39:09.480974 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.480828    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fba69b\"/\"test-model-fba69b-kserve-self-signed-certs\""
Jul 10 20:39:09.495357 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.495332    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"]
Jul 10 20:39:09.563643 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.563606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tmp-dir\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.563643 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.563642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tls-certs\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.563875 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.563706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-model-cache\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.563875 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.563762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-dshm\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.563875 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.563787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x779p\" (UniqueName: \"kubernetes.io/projected/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-kube-api-access-x779p\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.563875 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.563834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-home\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.664559 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-home\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.664747 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tmp-dir\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.664747 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tls-certs\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.664834 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664778    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-model-cache\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.664879 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-dshm\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.664921 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x779p\" (UniqueName: \"kubernetes.io/projected/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-kube-api-access-x779p\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.664921 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664902    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tmp-dir\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.665023 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.664986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-home\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.665084 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.665061    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-model-cache\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.666878 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.666856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-dshm\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.667204 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.667186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tls-certs\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.675741 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.675720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x779p\" (UniqueName: \"kubernetes.io/projected/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-kube-api-access-x779p\") pod \"test-model-fba69b-kserve-6d674994f7-w8wnw\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") " pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.784537 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.784496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:09.919369 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:09.919342    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"]
Jul 10 20:39:09.921726 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:39:09.921697    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3555e9f2_7eaa_4088_9eb7_7e3e7a076aba.slice/crio-f70b1c1029dd1a7c63e113708303b0d3a8b6b43bda089edc7a4f9ffc9d0412f0 WatchSource:0}: Error finding container f70b1c1029dd1a7c63e113708303b0d3a8b6b43bda089edc7a4f9ffc9d0412f0: Status 404 returned error can't find the container with id f70b1c1029dd1a7c63e113708303b0d3a8b6b43bda089edc7a4f9ffc9d0412f0
Jul 10 20:39:10.742936 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:10.742902    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw" event={"ID":"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba","Type":"ContainerStarted","Data":"19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081"}
Jul 10 20:39:10.742936 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:10.742944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw" event={"ID":"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba","Type":"ContainerStarted","Data":"f70b1c1029dd1a7c63e113708303b0d3a8b6b43bda089edc7a4f9ffc9d0412f0"}
Jul 10 20:39:10.763219 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:10.763169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw" podStartSLOduration=1.7631530720000002 podStartE2EDuration="1.763153072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:39:10.760568548 +0000 UTC m=+1837.986335469" watchObservedRunningTime="2026-07-10 20:39:10.763153072 +0000 UTC m=+1837.988920001"
Jul 10 20:39:19.785391 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:19.785356    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:19.785803 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:19.785408    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:19.798049 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:19.798024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:19.809107 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:19.809080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:32.029929 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:32.029897    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"]
Jul 10 20:39:32.832685 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:32.832632    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"]
Jul 10 20:39:32.832956 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:32.832901    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp" podUID="1cb42094-3bb5-4dd8-8884-9547b3a252b6" containerName="maas-api" containerID="cri-o://9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96" gracePeriod=30
Jul 10 20:39:33.082603 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.082580    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:39:33.175847 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.175748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb42094-3bb5-4dd8-8884-9547b3a252b6-openshift-service-ca\") pod \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") "
Jul 10 20:39:33.175847 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.175793    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzbww\" (UniqueName: \"kubernetes.io/projected/1cb42094-3bb5-4dd8-8884-9547b3a252b6-kube-api-access-dzbww\") pod \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") "
Jul 10 20:39:33.175847 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.175835    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls\") pod \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\" (UID: \"1cb42094-3bb5-4dd8-8884-9547b3a252b6\") "
Jul 10 20:39:33.176111 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.176086    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cb42094-3bb5-4dd8-8884-9547b3a252b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cb42094-3bb5-4dd8-8884-9547b3a252b6" (UID: "1cb42094-3bb5-4dd8-8884-9547b3a252b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:39:33.177998 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.177961    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cb42094-3bb5-4dd8-8884-9547b3a252b6-kube-api-access-dzbww" (OuterVolumeSpecName: "kube-api-access-dzbww") pod "1cb42094-3bb5-4dd8-8884-9547b3a252b6" (UID: "1cb42094-3bb5-4dd8-8884-9547b3a252b6"). InnerVolumeSpecName "kube-api-access-dzbww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:39:33.178105 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.177999    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cb42094-3bb5-4dd8-8884-9547b3a252b6" (UID: "1cb42094-3bb5-4dd8-8884-9547b3a252b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:39:33.277443 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.277411    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cb42094-3bb5-4dd8-8884-9547b3a252b6-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:33.277443 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.277442    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzbww\" (UniqueName: \"kubernetes.io/projected/1cb42094-3bb5-4dd8-8884-9547b3a252b6-kube-api-access-dzbww\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:33.277798 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.277454    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cb42094-3bb5-4dd8-8884-9547b3a252b6-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:33.865207 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.865174    2580 generic.go:356] "Generic (PLEG): container finished" podID="1cb42094-3bb5-4dd8-8884-9547b3a252b6" containerID="9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96" exitCode=0
Jul 10 20:39:33.865207 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.865212    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp" event={"ID":"1cb42094-3bb5-4dd8-8884-9547b3a252b6","Type":"ContainerDied","Data":"9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96"}
Jul 10 20:39:33.865416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.865218    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"
Jul 10 20:39:33.865416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.865252    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp" event={"ID":"1cb42094-3bb5-4dd8-8884-9547b3a252b6","Type":"ContainerDied","Data":"271293bbb6b5fce1611d537d626850808ca7a259551f44d4dd05c0ad47206697"}
Jul 10 20:39:33.865416 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.865273    2580 scope.go:117] "RemoveContainer" containerID="9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96"
Jul 10 20:39:33.875508 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.875491    2580 scope.go:117] "RemoveContainer" containerID="9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96"
Jul 10 20:39:33.875796 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:39:33.875771    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96\": container with ID starting with 9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96 not found: ID does not exist" containerID="9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96"
Jul 10 20:39:33.875905 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.875802    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96"} err="failed to get container status \"9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96\": rpc error: code = NotFound desc = could not find container \"9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96\": container with ID starting with 9d4cbef1f9514cee7178053c0b7fd8e7fe8b487e25005048ba9721a4cff2ee96 not found: ID does not exist"
Jul 10 20:39:33.884294 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.884268    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"]
Jul 10 20:39:33.887866 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:33.887840    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-58c4c1-67c55fbb5c-94jjp"]
Jul 10 20:39:35.280884 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:35.280850    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cb42094-3bb5-4dd8-8884-9547b3a252b6" path="/var/lib/kubelet/pods/1cb42094-3bb5-4dd8-8884-9547b3a252b6/volumes"
Jul 10 20:39:47.049743 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.049703    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw" podUID="3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" containerName="main" containerID="cri-o://19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081" gracePeriod=45
Jul 10 20:39:47.295019 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.294993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:47.409979 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.409886    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tmp-dir\") pod \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") "
Jul 10 20:39:47.409979 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.409950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tls-certs\") pod \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") "
Jul 10 20:39:47.410262 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.409998    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-model-cache\") pod \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") "
Jul 10 20:39:47.410262 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.410052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-home\") pod \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") "
Jul 10 20:39:47.410262 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.410102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x779p\" (UniqueName: \"kubernetes.io/projected/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-kube-api-access-x779p\") pod \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") "
Jul 10 20:39:47.410262 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.410128    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-dshm\") pod \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\" (UID: \"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba\") "
Jul 10 20:39:47.410262 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.410203    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" (UID: "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:39:47.410504 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.410374    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tmp-dir\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:47.410504 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.410395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-model-cache" (OuterVolumeSpecName: "model-cache") pod "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" (UID: "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:39:47.410504 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.410407    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-home" (OuterVolumeSpecName: "home") pod "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" (UID: "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:39:47.412343 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.412316    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-dshm" (OuterVolumeSpecName: "dshm") pod "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" (UID: "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:39:47.412458 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.412389    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" (UID: "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:39:47.412458 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.412399    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-kube-api-access-x779p" (OuterVolumeSpecName: "kube-api-access-x779p") pod "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" (UID: "3555e9f2-7eaa-4088-9eb7-7e3e7a076aba"). InnerVolumeSpecName "kube-api-access-x779p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:39:47.510932 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.510895    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-home\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:47.510932 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.510925    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x779p\" (UniqueName: \"kubernetes.io/projected/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-kube-api-access-x779p\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:47.510932 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.510939    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-dshm\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:47.511172 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.510951    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-tls-certs\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:47.511172 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.510977    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba-model-cache\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:39:47.936475 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.936441    2580 generic.go:356] "Generic (PLEG): container finished" podID="3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" containerID="19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081" exitCode=0
Jul 10 20:39:47.936652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.936487    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"
Jul 10 20:39:47.936652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.936528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw" event={"ID":"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba","Type":"ContainerDied","Data":"19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081"}
Jul 10 20:39:47.936652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.936577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw" event={"ID":"3555e9f2-7eaa-4088-9eb7-7e3e7a076aba","Type":"ContainerDied","Data":"f70b1c1029dd1a7c63e113708303b0d3a8b6b43bda089edc7a4f9ffc9d0412f0"}
Jul 10 20:39:47.936652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.936595    2580 scope.go:117] "RemoveContainer" containerID="19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081"
Jul 10 20:39:47.945643 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.945628    2580 scope.go:117] "RemoveContainer" containerID="19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081"
Jul 10 20:39:47.945950 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:39:47.945929    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081\": container with ID starting with 19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081 not found: ID does not exist" containerID="19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081"
Jul 10 20:39:47.946017 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.945958    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081"} err="failed to get container status \"19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081\": rpc error: code = NotFound desc = could not find container \"19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081\": container with ID starting with 19a24a799b3a5511574f3f99d3282d5b3ee9efad18cd2742e57364ea94a4a081 not found: ID does not exist"
Jul 10 20:39:47.957818 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.957795    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"]
Jul 10 20:39:47.961610 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:47.961585    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-fba69b/test-model-fba69b-kserve-6d674994f7-w8wnw"]
Jul 10 20:39:49.280479 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:39:49.280446    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" path="/var/lib/kubelet/pods/3555e9f2-7eaa-4088-9eb7-7e3e7a076aba/volumes"
Jul 10 20:40:15.823980 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:15.823944    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"]
Jul 10 20:40:15.824389 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:15.824209    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r" podUID="f216d124-c412-4e60-8df5-7ce224e00941" containerName="maas-api" containerID="cri-o://726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a" gracePeriod=30
Jul 10 20:40:17.073522 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.073498    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:40:17.096608 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.096574    2580 generic.go:356] "Generic (PLEG): container finished" podID="f216d124-c412-4e60-8df5-7ce224e00941" containerID="726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a" exitCode=0
Jul 10 20:40:17.096608 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.096609    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"
Jul 10 20:40:17.096831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.096645    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r" event={"ID":"f216d124-c412-4e60-8df5-7ce224e00941","Type":"ContainerDied","Data":"726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a"}
Jul 10 20:40:17.096831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.096711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r" event={"ID":"f216d124-c412-4e60-8df5-7ce224e00941","Type":"ContainerDied","Data":"89f5bf0c092832905fa40e95b6c2fdd9664d69fc20a6acf0c4a078e4f5af6fd1"}
Jul 10 20:40:17.096831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.096734    2580 scope.go:117] "RemoveContainer" containerID="726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a"
Jul 10 20:40:17.105189 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.105174    2580 scope.go:117] "RemoveContainer" containerID="726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a"
Jul 10 20:40:17.105434 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:40:17.105414    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a\": container with ID starting with 726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a not found: ID does not exist" containerID="726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a"
Jul 10 20:40:17.105496 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.105443    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a"} err="failed to get container status \"726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a\": rpc error: code = NotFound desc = could not find container \"726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a\": container with ID starting with 726ae83767bdd5b7051d542524607a79c0980e83d334784abb9e6cb51c095e1a not found: ID does not exist"
Jul 10 20:40:17.171446 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.171416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls\") pod \"f216d124-c412-4e60-8df5-7ce224e00941\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") "
Jul 10 20:40:17.171614 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.171459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f216d124-c412-4e60-8df5-7ce224e00941-openshift-service-ca\") pod \"f216d124-c412-4e60-8df5-7ce224e00941\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") "
Jul 10 20:40:17.171614 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.171488    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwjf7\" (UniqueName: \"kubernetes.io/projected/f216d124-c412-4e60-8df5-7ce224e00941-kube-api-access-wwjf7\") pod \"f216d124-c412-4e60-8df5-7ce224e00941\" (UID: \"f216d124-c412-4e60-8df5-7ce224e00941\") "
Jul 10 20:40:17.171831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.171809    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f216d124-c412-4e60-8df5-7ce224e00941-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f216d124-c412-4e60-8df5-7ce224e00941" (UID: "f216d124-c412-4e60-8df5-7ce224e00941"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:40:17.173626 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.173559    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f216d124-c412-4e60-8df5-7ce224e00941" (UID: "f216d124-c412-4e60-8df5-7ce224e00941"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:40:17.173626 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.173599    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f216d124-c412-4e60-8df5-7ce224e00941-kube-api-access-wwjf7" (OuterVolumeSpecName: "kube-api-access-wwjf7") pod "f216d124-c412-4e60-8df5-7ce224e00941" (UID: "f216d124-c412-4e60-8df5-7ce224e00941"). InnerVolumeSpecName "kube-api-access-wwjf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:40:17.272168 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.272132    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f216d124-c412-4e60-8df5-7ce224e00941-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:17.272168 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.272161    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f216d124-c412-4e60-8df5-7ce224e00941-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:17.272168 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.272172    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwjf7\" (UniqueName: \"kubernetes.io/projected/f216d124-c412-4e60-8df5-7ce224e00941-kube-api-access-wwjf7\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:17.414417 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.414384    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"]
Jul 10 20:40:17.417859 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:17.417836    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fba69b-89dc4cd48-8qx5r"]
Jul 10 20:40:19.281913 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:19.281882    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f216d124-c412-4e60-8df5-7ce224e00941" path="/var/lib/kubelet/pods/f216d124-c412-4e60-8df5-7ce224e00941/volumes"
Jul 10 20:40:25.291420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291386    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"]
Jul 10 20:40:25.291892 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291820    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" containerName="main"
Jul 10 20:40:25.291892 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291835    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" containerName="main"
Jul 10 20:40:25.291892 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291848    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cb42094-3bb5-4dd8-8884-9547b3a252b6" containerName="maas-api"
Jul 10 20:40:25.291892 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291854    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cb42094-3bb5-4dd8-8884-9547b3a252b6" containerName="maas-api"
Jul 10 20:40:25.291892 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291872    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f216d124-c412-4e60-8df5-7ce224e00941" containerName="maas-api"
Jul 10 20:40:25.291892 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291880    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f216d124-c412-4e60-8df5-7ce224e00941" containerName="maas-api"
Jul 10 20:40:25.292203 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291937    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cb42094-3bb5-4dd8-8884-9547b3a252b6" containerName="maas-api"
Jul 10 20:40:25.292203 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291945    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f216d124-c412-4e60-8df5-7ce224e00941" containerName="maas-api"
Jul 10 20:40:25.292203 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.291958    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3555e9f2-7eaa-4088-9eb7-7e3e7a076aba" containerName="main"
Jul 10 20:40:25.296395 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.296376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.298780 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.298758    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-3ab324-serving-cert\""
Jul 10 20:40:25.307907 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.307885    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"]
Jul 10 20:40:25.345816 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.345781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/967cc977-cf55-42fc-8df6-c4fcd6ce7240-openshift-service-ca\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.345816 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.345815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbc2q\" (UniqueName: \"kubernetes.io/projected/967cc977-cf55-42fc-8df6-c4fcd6ce7240-kube-api-access-sbc2q\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.346006 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.345878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/967cc977-cf55-42fc-8df6-c4fcd6ce7240-maas-api-tls\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.446735 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.446698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/967cc977-cf55-42fc-8df6-c4fcd6ce7240-maas-api-tls\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.446948 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.446790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/967cc977-cf55-42fc-8df6-c4fcd6ce7240-openshift-service-ca\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.446948 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.446812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbc2q\" (UniqueName: \"kubernetes.io/projected/967cc977-cf55-42fc-8df6-c4fcd6ce7240-kube-api-access-sbc2q\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.447475 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.447454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/967cc977-cf55-42fc-8df6-c4fcd6ce7240-openshift-service-ca\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.449230 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.449206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/967cc977-cf55-42fc-8df6-c4fcd6ce7240-maas-api-tls\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.455238 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.455215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbc2q\" (UniqueName: \"kubernetes.io/projected/967cc977-cf55-42fc-8df6-c4fcd6ce7240-kube-api-access-sbc2q\") pod \"maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.608740 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.608617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:25.747799 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:25.747769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"]
Jul 10 20:40:25.749094 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:40:25.749066    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967cc977_cf55_42fc_8df6_c4fcd6ce7240.slice/crio-fe1b7319d7d9cdafa0b03f1375c3fb89972b067fb805c14ef9f0e3e5f158f8a4 WatchSource:0}: Error finding container fe1b7319d7d9cdafa0b03f1375c3fb89972b067fb805c14ef9f0e3e5f158f8a4: Status 404 returned error can't find the container with id fe1b7319d7d9cdafa0b03f1375c3fb89972b067fb805c14ef9f0e3e5f158f8a4
Jul 10 20:40:26.145447 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:26.145361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t" event={"ID":"967cc977-cf55-42fc-8df6-c4fcd6ce7240","Type":"ContainerStarted","Data":"fe1b7319d7d9cdafa0b03f1375c3fb89972b067fb805c14ef9f0e3e5f158f8a4"}
Jul 10 20:40:27.153014 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:27.152971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t" event={"ID":"967cc977-cf55-42fc-8df6-c4fcd6ce7240","Type":"ContainerStarted","Data":"164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475"}
Jul 10 20:40:27.153503 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:27.153039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:27.171255 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:27.171201    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t" podStartSLOduration=1.65909034 podStartE2EDuration="2.171186509s" totalImagesPullingTime="512.096169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:40:27.168369659 +0000 UTC m=+1914.394136612" watchObservedRunningTime="2026-07-10 20:40:27.171186509 +0000 UTC m=+1914.396953439"
Jul 10 20:40:33.163957 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.163927    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:33.353446 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.353391    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"]
Jul 10 20:40:33.357351 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.357327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.359922 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.359898    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-562022-serving-cert\""
Jul 10 20:40:33.369495 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.369471    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"]
Jul 10 20:40:33.415941 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.415912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0e951c9-c6b3-4088-b420-a6a1a4614019-openshift-service-ca\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.416043 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.415971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzpx8\" (UniqueName: \"kubernetes.io/projected/b0e951c9-c6b3-4088-b420-a6a1a4614019-kube-api-access-dzpx8\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.416043 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.416000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.516472 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.516434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.516656 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.516518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0e951c9-c6b3-4088-b420-a6a1a4614019-openshift-service-ca\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.516656 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.516559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzpx8\" (UniqueName: \"kubernetes.io/projected/b0e951c9-c6b3-4088-b420-a6a1a4614019-kube-api-access-dzpx8\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.516656 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:40:33.516592    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-562022-serving-cert: secret "maas-api-e2e-api-b-562022-serving-cert" not found
Jul 10 20:40:33.516877 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:40:33.516701    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls podName:b0e951c9-c6b3-4088-b420-a6a1a4614019 nodeName:}" failed. No retries permitted until 2026-07-10 20:40:34.016655624 +0000 UTC m=+1921.242422531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls") pod "maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56" (UID: "b0e951c9-c6b3-4088-b420-a6a1a4614019") : secret "maas-api-e2e-api-b-562022-serving-cert" not found
Jul 10 20:40:33.517247 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.517220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0e951c9-c6b3-4088-b420-a6a1a4614019-openshift-service-ca\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:33.527398 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:33.527370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzpx8\" (UniqueName: \"kubernetes.io/projected/b0e951c9-c6b3-4088-b420-a6a1a4614019-kube-api-access-dzpx8\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:34.021128 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:34.021089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:34.023820 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:34.023795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls\") pod \"maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:34.269835 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:34.269800    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:34.410766 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:34.410731    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"]
Jul 10 20:40:35.195765 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:35.195656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56" event={"ID":"b0e951c9-c6b3-4088-b420-a6a1a4614019","Type":"ContainerStarted","Data":"9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb"}
Jul 10 20:40:35.195765 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:35.195747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56" event={"ID":"b0e951c9-c6b3-4088-b420-a6a1a4614019","Type":"ContainerStarted","Data":"e87f30ca523da3831f12fdc5256ffd1ebff957f3f273c460644f23e92f96bbb4"}
Jul 10 20:40:35.195943 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:35.195790    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:35.212884 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:35.212789    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56" podStartSLOduration=1.833782187 podStartE2EDuration="2.212772462s" totalImagesPullingTime="378.990275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:40:35.212217489 +0000 UTC m=+1922.437984421" watchObservedRunningTime="2026-07-10 20:40:35.212772462 +0000 UTC m=+1922.438539392"
Jul 10 20:40:41.206193 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:41.206161    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:46.664158 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:46.664112    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"]
Jul 10 20:40:46.664572 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:46.664387    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t" podUID="967cc977-cf55-42fc-8df6-c4fcd6ce7240" containerName="maas-api" containerID="cri-o://164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475" gracePeriod=30
Jul 10 20:40:46.917518 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:46.917457    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:47.045888 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.045851    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbc2q\" (UniqueName: \"kubernetes.io/projected/967cc977-cf55-42fc-8df6-c4fcd6ce7240-kube-api-access-sbc2q\") pod \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") "
Jul 10 20:40:47.046059 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.045945    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/967cc977-cf55-42fc-8df6-c4fcd6ce7240-openshift-service-ca\") pod \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") "
Jul 10 20:40:47.046059 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.045982    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/967cc977-cf55-42fc-8df6-c4fcd6ce7240-maas-api-tls\") pod \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\" (UID: \"967cc977-cf55-42fc-8df6-c4fcd6ce7240\") "
Jul 10 20:40:47.046339 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.046312    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/967cc977-cf55-42fc-8df6-c4fcd6ce7240-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "967cc977-cf55-42fc-8df6-c4fcd6ce7240" (UID: "967cc977-cf55-42fc-8df6-c4fcd6ce7240"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:40:47.048154 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.048116    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/967cc977-cf55-42fc-8df6-c4fcd6ce7240-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "967cc977-cf55-42fc-8df6-c4fcd6ce7240" (UID: "967cc977-cf55-42fc-8df6-c4fcd6ce7240"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:40:47.048154 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.048148    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/967cc977-cf55-42fc-8df6-c4fcd6ce7240-kube-api-access-sbc2q" (OuterVolumeSpecName: "kube-api-access-sbc2q") pod "967cc977-cf55-42fc-8df6-c4fcd6ce7240" (UID: "967cc977-cf55-42fc-8df6-c4fcd6ce7240"). InnerVolumeSpecName "kube-api-access-sbc2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:40:47.147460 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.147424    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbc2q\" (UniqueName: \"kubernetes.io/projected/967cc977-cf55-42fc-8df6-c4fcd6ce7240-kube-api-access-sbc2q\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:47.147460 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.147453    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/967cc977-cf55-42fc-8df6-c4fcd6ce7240-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:47.147460 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.147464    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/967cc977-cf55-42fc-8df6-c4fcd6ce7240-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:47.264908 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.264816    2580 generic.go:356] "Generic (PLEG): container finished" podID="967cc977-cf55-42fc-8df6-c4fcd6ce7240" containerID="164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475" exitCode=0
Jul 10 20:40:47.264908 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.264871    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"
Jul 10 20:40:47.264908 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.264900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t" event={"ID":"967cc977-cf55-42fc-8df6-c4fcd6ce7240","Type":"ContainerDied","Data":"164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475"}
Jul 10 20:40:47.265185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.264952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t" event={"ID":"967cc977-cf55-42fc-8df6-c4fcd6ce7240","Type":"ContainerDied","Data":"fe1b7319d7d9cdafa0b03f1375c3fb89972b067fb805c14ef9f0e3e5f158f8a4"}
Jul 10 20:40:47.265185 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.264978    2580 scope.go:117] "RemoveContainer" containerID="164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475"
Jul 10 20:40:47.274426 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.274409    2580 scope.go:117] "RemoveContainer" containerID="164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475"
Jul 10 20:40:47.274713 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:40:47.274685    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475\": container with ID starting with 164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475 not found: ID does not exist" containerID="164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475"
Jul 10 20:40:47.274809 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.274722    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475"} err="failed to get container status \"164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475\": rpc error: code = NotFound desc = could not find container \"164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475\": container with ID starting with 164d303b9abe62ddc108ba86440e862443360091af8332de2f8271803be2c475 not found: ID does not exist"
Jul 10 20:40:47.287172 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.287150    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"]
Jul 10 20:40:47.291811 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:47.291790    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3ab324-7b8d5f5d95-8tc7t"]
Jul 10 20:40:49.280423 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:49.280391    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="967cc977-cf55-42fc-8df6-c4fcd6ce7240" path="/var/lib/kubelet/pods/967cc977-cf55-42fc-8df6-c4fcd6ce7240/volumes"
Jul 10 20:40:53.703918 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:53.703878    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"]
Jul 10 20:40:53.704504 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:53.704161    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56" podUID="b0e951c9-c6b3-4088-b420-a6a1a4614019" containerName="maas-api" containerID="cri-o://9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb" gracePeriod=30
Jul 10 20:40:53.951771 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:53.951747    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:54.110139 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.110111    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0e951c9-c6b3-4088-b420-a6a1a4614019-openshift-service-ca\") pod \"b0e951c9-c6b3-4088-b420-a6a1a4614019\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") "
Jul 10 20:40:54.110139 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.110142    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls\") pod \"b0e951c9-c6b3-4088-b420-a6a1a4614019\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") "
Jul 10 20:40:54.110363 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.110191    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzpx8\" (UniqueName: \"kubernetes.io/projected/b0e951c9-c6b3-4088-b420-a6a1a4614019-kube-api-access-dzpx8\") pod \"b0e951c9-c6b3-4088-b420-a6a1a4614019\" (UID: \"b0e951c9-c6b3-4088-b420-a6a1a4614019\") "
Jul 10 20:40:54.110706 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.110653    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0e951c9-c6b3-4088-b420-a6a1a4614019-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0e951c9-c6b3-4088-b420-a6a1a4614019" (UID: "b0e951c9-c6b3-4088-b420-a6a1a4614019"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:40:54.112120 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.112094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0e951c9-c6b3-4088-b420-a6a1a4614019" (UID: "b0e951c9-c6b3-4088-b420-a6a1a4614019"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:40:54.112203 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.112153    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e951c9-c6b3-4088-b420-a6a1a4614019-kube-api-access-dzpx8" (OuterVolumeSpecName: "kube-api-access-dzpx8") pod "b0e951c9-c6b3-4088-b420-a6a1a4614019" (UID: "b0e951c9-c6b3-4088-b420-a6a1a4614019"). InnerVolumeSpecName "kube-api-access-dzpx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:40:54.211831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.211792    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0e951c9-c6b3-4088-b420-a6a1a4614019-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:54.211831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.211823    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0e951c9-c6b3-4088-b420-a6a1a4614019-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:54.211831 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.211833    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzpx8\" (UniqueName: \"kubernetes.io/projected/b0e951c9-c6b3-4088-b420-a6a1a4614019-kube-api-access-dzpx8\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:40:54.308068 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.308032    2580 generic.go:356] "Generic (PLEG): container finished" podID="b0e951c9-c6b3-4088-b420-a6a1a4614019" containerID="9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb" exitCode=0
Jul 10 20:40:54.308233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.308080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"
Jul 10 20:40:54.308233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.308119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56" event={"ID":"b0e951c9-c6b3-4088-b420-a6a1a4614019","Type":"ContainerDied","Data":"9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb"}
Jul 10 20:40:54.308233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.308172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56" event={"ID":"b0e951c9-c6b3-4088-b420-a6a1a4614019","Type":"ContainerDied","Data":"e87f30ca523da3831f12fdc5256ffd1ebff957f3f273c460644f23e92f96bbb4"}
Jul 10 20:40:54.308233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.308195    2580 scope.go:117] "RemoveContainer" containerID="9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb"
Jul 10 20:40:54.316809 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.316785    2580 scope.go:117] "RemoveContainer" containerID="9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb"
Jul 10 20:40:54.317060 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:40:54.317037    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb\": container with ID starting with 9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb not found: ID does not exist" containerID="9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb"
Jul 10 20:40:54.317107 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.317067    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb"} err="failed to get container status \"9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb\": rpc error: code = NotFound desc = could not find container \"9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb\": container with ID starting with 9a8ac338d63182064206c44cfb4366dfe3d706191a39f40c7a59c85bbe83b4fb not found: ID does not exist"
Jul 10 20:40:54.328994 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.328970    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"]
Jul 10 20:40:54.332310 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:54.332290    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-562022-7bdd8d5ff6-sps56"]
Jul 10 20:40:55.281531 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:40:55.281501    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0e951c9-c6b3-4088-b420-a6a1a4614019" path="/var/lib/kubelet/pods/b0e951c9-c6b3-4088-b420-a6a1a4614019/volumes"
Jul 10 20:41:03.698536 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.698494    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"]
Jul 10 20:41:03.699178 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.699051    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0e951c9-c6b3-4088-b420-a6a1a4614019" containerName="maas-api"
Jul 10 20:41:03.699178 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.699070    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0e951c9-c6b3-4088-b420-a6a1a4614019" containerName="maas-api"
Jul 10 20:41:03.699178 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.699105    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="967cc977-cf55-42fc-8df6-c4fcd6ce7240" containerName="maas-api"
Jul 10 20:41:03.699178 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.699114    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="967cc977-cf55-42fc-8df6-c4fcd6ce7240" containerName="maas-api"
Jul 10 20:41:03.699357 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.699202    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0e951c9-c6b3-4088-b420-a6a1a4614019" containerName="maas-api"
Jul 10 20:41:03.699357 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.699215    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="967cc977-cf55-42fc-8df6-c4fcd6ce7240" containerName="maas-api"
Jul 10 20:41:03.703838 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.703821    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.706177 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.706159    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-75e636-serving-cert\""
Jul 10 20:41:03.721315 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.721277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"]
Jul 10 20:41:03.780206 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.780174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-openshift-service-ca\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.780371 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.780222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm9mx\" (UniqueName: \"kubernetes.io/projected/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-kube-api-access-wm9mx\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.780371 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.780266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.881052 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.881007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm9mx\" (UniqueName: \"kubernetes.io/projected/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-kube-api-access-wm9mx\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.881267 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.881086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.881267 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.881138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-openshift-service-ca\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.881267 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:03.881236    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-serving-cert: secret "maas-api-e2e-shared-a-75e636-serving-cert" not found
Jul 10 20:41:03.881441 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:03.881310    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls podName:b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:04.381291535 +0000 UTC m=+1951.607058445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls") pod "maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" (UID: "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588") : secret "maas-api-e2e-shared-a-75e636-serving-cert" not found
Jul 10 20:41:03.881763 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.881740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-openshift-service-ca\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:03.892788 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:03.892758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm9mx\" (UniqueName: \"kubernetes.io/projected/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-kube-api-access-wm9mx\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:04.384576 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:04.384547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:04.384769 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:04.384680    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-serving-cert: secret "maas-api-e2e-shared-a-75e636-serving-cert" not found
Jul 10 20:41:04.384769 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:04.384741    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls podName:b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588 nodeName:}" failed. No retries permitted until 2026-07-10 20:41:05.384725372 +0000 UTC m=+1952.610492279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls") pod "maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" (UID: "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588") : secret "maas-api-e2e-shared-a-75e636-serving-cert" not found
Jul 10 20:41:05.392405 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:05.392368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:05.394697 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:05.394662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls\") pod \"maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:05.519755 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:05.519718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:05.654873 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:05.654842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"]
Jul 10 20:41:05.656191 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:41:05.656167    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2efbdbc_ca51_48d8_84d7_a2d2d8b1d588.slice/crio-7bd0d2c7e42c8e5635e112adadcd0888b7377f1591a6a165553730949f485977 WatchSource:0}: Error finding container 7bd0d2c7e42c8e5635e112adadcd0888b7377f1591a6a165553730949f485977: Status 404 returned error can't find the container with id 7bd0d2c7e42c8e5635e112adadcd0888b7377f1591a6a165553730949f485977
Jul 10 20:41:06.372888 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:06.372855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" event={"ID":"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588","Type":"ContainerStarted","Data":"2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d"}
Jul 10 20:41:06.373055 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:06.372907    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" event={"ID":"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588","Type":"ContainerStarted","Data":"7bd0d2c7e42c8e5635e112adadcd0888b7377f1591a6a165553730949f485977"}
Jul 10 20:41:06.373055 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:06.372935    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:06.389448 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:06.389400    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" podStartSLOduration=2.8736854689999998 podStartE2EDuration="3.38938459s" totalImagesPullingTime="515.699121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:06.388008094 +0000 UTC m=+1953.613775023" watchObservedRunningTime="2026-07-10 20:41:06.38938459 +0000 UTC m=+1953.615151518"
Jul 10 20:41:11.572392 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.572356    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"]
Jul 10 20:41:11.576046 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.576028    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.578581 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.578557    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-1b46b3-serving-cert\""
Jul 10 20:41:11.587432 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.587396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"]
Jul 10 20:41:11.639399 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.639364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx8pq\" (UniqueName: \"kubernetes.io/projected/596240c9-9547-4f6b-b3b4-80e88dadff7c-kube-api-access-hx8pq\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.639564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.639413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.639564 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.639532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596240c9-9547-4f6b-b3b4-80e88dadff7c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.740159 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.740120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx8pq\" (UniqueName: \"kubernetes.io/projected/596240c9-9547-4f6b-b3b4-80e88dadff7c-kube-api-access-hx8pq\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.740343 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.740169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.740343 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.740228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596240c9-9547-4f6b-b3b4-80e88dadff7c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.740434 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:11.740336    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-serving-cert: secret "maas-api-e2e-shared-b-1b46b3-serving-cert" not found
Jul 10 20:41:11.740434 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:11.740411    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls podName:596240c9-9547-4f6b-b3b4-80e88dadff7c nodeName:}" failed. No retries permitted until 2026-07-10 20:41:12.240388749 +0000 UTC m=+1959.466155662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls") pod "maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" (UID: "596240c9-9547-4f6b-b3b4-80e88dadff7c") : secret "maas-api-e2e-shared-b-1b46b3-serving-cert" not found
Jul 10 20:41:11.740929 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.740912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596240c9-9547-4f6b-b3b4-80e88dadff7c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:11.760129 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:11.760099    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx8pq\" (UniqueName: \"kubernetes.io/projected/596240c9-9547-4f6b-b3b4-80e88dadff7c-kube-api-access-hx8pq\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:12.244761 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:12.244728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:12.244943 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:12.244876    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-serving-cert: secret "maas-api-e2e-shared-b-1b46b3-serving-cert" not found
Jul 10 20:41:12.245000 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:41:12.244955    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls podName:596240c9-9547-4f6b-b3b4-80e88dadff7c nodeName:}" failed. No retries permitted until 2026-07-10 20:41:13.244938865 +0000 UTC m=+1960.470705772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls") pod "maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" (UID: "596240c9-9547-4f6b-b3b4-80e88dadff7c") : secret "maas-api-e2e-shared-b-1b46b3-serving-cert" not found
Jul 10 20:41:12.383810 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:12.383779    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:41:13.255327 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:13.255280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:13.257872 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:13.257850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls\") pod \"maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:13.388960 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:13.388923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:13.570913 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:13.570887    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"]
Jul 10 20:41:13.573153 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:41:13.573120    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596240c9_9547_4f6b_b3b4_80e88dadff7c.slice/crio-c934becb3e9374478287e950c8c59d80878e40b52594f446e63f74dd2593c3a8 WatchSource:0}: Error finding container c934becb3e9374478287e950c8c59d80878e40b52594f446e63f74dd2593c3a8: Status 404 returned error can't find the container with id c934becb3e9374478287e950c8c59d80878e40b52594f446e63f74dd2593c3a8
Jul 10 20:41:14.419785 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:14.419748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" event={"ID":"596240c9-9547-4f6b-b3b4-80e88dadff7c","Type":"ContainerStarted","Data":"e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097"}
Jul 10 20:41:14.420169 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:14.419804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" event={"ID":"596240c9-9547-4f6b-b3b4-80e88dadff7c","Type":"ContainerStarted","Data":"c934becb3e9374478287e950c8c59d80878e40b52594f446e63f74dd2593c3a8"}
Jul 10 20:41:14.420169 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:14.419827    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:41:14.454640 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:14.454587    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" podStartSLOduration=3.048533441 podStartE2EDuration="3.454568981s" totalImagesPullingTime="406.03554ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:41:14.449875522 +0000 UTC m=+1961.675642450" watchObservedRunningTime="2026-07-10 20:41:14.454568981 +0000 UTC m=+1961.680335910"
Jul 10 20:41:20.430721 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:41:20.430694    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:43:33.347514 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:43:33.347486    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:43:33.351756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:43:33.351740    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dqm9m_6f224662-2d68-4d3a-bb00-16fa779f6af4/ovn-acl-logging/0.log"
Jul 10 20:44:31.948728 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:31.948690    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"]
Jul 10 20:44:31.952204 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:31.952188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:31.955963 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:31.955935    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-75e636\"/\"kube-root-ca.crt\""
Jul 10 20:44:31.955963 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:31.955952    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-75e636\"/\"openshift-service-ca.crt\""
Jul 10 20:44:31.955963 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:31.955965    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-75e636\"/\"default-dockercfg-qlpc9\""
Jul 10 20:44:31.956200 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:31.955979    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-75e636\"/\"sub-test-model-75e636-kserve-self-signed-certs\""
Jul 10 20:44:31.962928 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:31.962906    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"]
Jul 10 20:44:32.010151 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.010119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52xhm\" (UniqueName: \"kubernetes.io/projected/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-kube-api-access-52xhm\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.010385 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.010360    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-home\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.010537 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.010408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-model-cache\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.010537 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.010436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tmp-dir\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.010537 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.010475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-dshm\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.010789 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.010616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tls-certs\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.111516 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.111477    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52xhm\" (UniqueName: \"kubernetes.io/projected/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-kube-api-access-52xhm\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.111742 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.111532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-home\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.111742 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.111567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-model-cache\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.111742 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.111587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tmp-dir\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.111742 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.111616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-dshm\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.111742 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.111661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tls-certs\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.112113 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.112089    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-model-cache\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.112191 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.112132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-home\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.112191 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.112162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tmp-dir\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.114020 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.113999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-dshm\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.114285 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.114267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tls-certs\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.119699 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.119664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52xhm\" (UniqueName: \"kubernetes.io/projected/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-kube-api-access-52xhm\") pod \"sub-test-model-75e636-kserve-85598796cf-4n7w5\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") " pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.261794 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.261700    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:32.398696 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.398657    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"]
Jul 10 20:44:32.400336 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:44:32.400306    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa73ed9c_3d4c_4fef_ad15_59a7dc7baa5e.slice/crio-e8bc059e8638363e338f6ab7093c81e402ad8ac7b5d21e9cd98671ded076fdb6 WatchSource:0}: Error finding container e8bc059e8638363e338f6ab7093c81e402ad8ac7b5d21e9cd98671ded076fdb6: Status 404 returned error can't find the container with id e8bc059e8638363e338f6ab7093c81e402ad8ac7b5d21e9cd98671ded076fdb6
Jul 10 20:44:32.449458 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:32.449430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5" event={"ID":"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e","Type":"ContainerStarted","Data":"e8bc059e8638363e338f6ab7093c81e402ad8ac7b5d21e9cd98671ded076fdb6"}
Jul 10 20:44:33.457192 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:33.457154    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5" event={"ID":"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e","Type":"ContainerStarted","Data":"f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69"}
Jul 10 20:44:33.475089 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:33.475042    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5" podStartSLOduration=2.4750272779999998 podStartE2EDuration="2.475027278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 20:44:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:44:33.472854127 +0000 UTC m=+2160.698621072" watchObservedRunningTime="2026-07-10 20:44:33.475027278 +0000 UTC m=+2160.700794206"
Jul 10 20:44:42.262425 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:42.262392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:42.262845 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:42.262511    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:42.274954 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:42.274928    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:44:42.516420 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:44:42.516335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:45:43.540168 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:43.540128    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"]
Jul 10 20:45:58.560895 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.560851    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5" podUID="fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" containerName="main" containerID="cri-o://f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69" gracePeriod=45
Jul 10 20:45:58.805822 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.805800    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:45:58.913973 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.913883    2580 generic.go:356] "Generic (PLEG): container finished" podID="fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" containerID="f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69" exitCode=0
Jul 10 20:45:58.913973 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.913922    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"
Jul 10 20:45:58.913973 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.913963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5" event={"ID":"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e","Type":"ContainerDied","Data":"f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69"}
Jul 10 20:45:58.914232 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.914006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5" event={"ID":"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e","Type":"ContainerDied","Data":"e8bc059e8638363e338f6ab7093c81e402ad8ac7b5d21e9cd98671ded076fdb6"}
Jul 10 20:45:58.914232 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.914025    2580 scope.go:117] "RemoveContainer" containerID="f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69"
Jul 10 20:45:58.922525 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.922508    2580 scope.go:117] "RemoveContainer" containerID="f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69"
Jul 10 20:45:58.922806 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:45:58.922788    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69\": container with ID starting with f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69 not found: ID does not exist" containerID="f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69"
Jul 10 20:45:58.922867 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.922820    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69"} err="failed to get container status \"f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69\": rpc error: code = NotFound desc = could not find container \"f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69\": container with ID starting with f2fba71f9e4d446e0516b1bdb70ac186cee0881f4f028d7274a59fe78ab85b69 not found: ID does not exist"
Jul 10 20:45:58.989268 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989230    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-home\") pod \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") "
Jul 10 20:45:58.989268 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989273    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tmp-dir\") pod \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") "
Jul 10 20:45:58.989501 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989304    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52xhm\" (UniqueName: \"kubernetes.io/projected/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-kube-api-access-52xhm\") pod \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") "
Jul 10 20:45:58.989501 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989334    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-model-cache\") pod \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") "
Jul 10 20:45:58.989501 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989376    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tls-certs\") pod \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") "
Jul 10 20:45:58.989501 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-dshm\") pod \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\" (UID: \"fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e\") "
Jul 10 20:45:58.989706 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989538    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" (UID: "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:58.989706 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989556    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-home" (OuterVolumeSpecName: "home") pod "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" (UID: "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:58.989706 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.989607    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" (UID: "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:58.996700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.990414    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-home\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:45:58.996700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.990443    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tmp-dir\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:45:58.996700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.990458    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-model-cache\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:45:58.997643 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.997608    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-dshm" (OuterVolumeSpecName: "dshm") pod "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" (UID: "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 20:45:58.997643 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.997614    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" (UID: "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:45:58.997643 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:58.997610    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-kube-api-access-52xhm" (OuterVolumeSpecName: "kube-api-access-52xhm") pod "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" (UID: "fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e"). InnerVolumeSpecName "kube-api-access-52xhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:45:59.091035 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:59.091005    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-tls-certs\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:45:59.091035 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:59.091029    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-dshm\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:45:59.091035 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:59.091039    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52xhm\" (UniqueName: \"kubernetes.io/projected/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e-kube-api-access-52xhm\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:45:59.236196 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:59.236168    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"]
Jul 10 20:45:59.239884 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:59.239861    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-75e636/sub-test-model-75e636-kserve-85598796cf-4n7w5"]
Jul 10 20:45:59.281128 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:45:59.281099    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" path="/var/lib/kubelet/pods/fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e/volumes"
Jul 10 20:47:18.704623 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.704588    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"]
Jul 10 20:47:18.705178 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.704818    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" podUID="b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" containerName="maas-api" containerID="cri-o://2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d" gracePeriod=30
Jul 10 20:47:18.957454 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.957390    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:47:18.995025 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.994988    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls\") pod \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") "
Jul 10 20:47:18.995189 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.995106    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm9mx\" (UniqueName: \"kubernetes.io/projected/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-kube-api-access-wm9mx\") pod \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") "
Jul 10 20:47:18.995189 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.995168    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-openshift-service-ca\") pod \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\" (UID: \"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588\") "
Jul 10 20:47:18.995521 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.995495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" (UID: "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:47:18.997139 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.997105    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" (UID: "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:47:18.997139 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:18.997131    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-kube-api-access-wm9mx" (OuterVolumeSpecName: "kube-api-access-wm9mx") pod "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" (UID: "b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588"). InnerVolumeSpecName "kube-api-access-wm9mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:47:19.096766 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.096729    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:47:19.096766 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.096758    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm9mx\" (UniqueName: \"kubernetes.io/projected/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-kube-api-access-wm9mx\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:47:19.096766 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.096767    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:47:19.337760 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.337724    2580 generic.go:356] "Generic (PLEG): container finished" podID="b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" containerID="2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d" exitCode=0
Jul 10 20:47:19.337935 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.337775    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"
Jul 10 20:47:19.337935 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.337808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" event={"ID":"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588","Type":"ContainerDied","Data":"2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d"}
Jul 10 20:47:19.337935 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.337860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc" event={"ID":"b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588","Type":"ContainerDied","Data":"7bd0d2c7e42c8e5635e112adadcd0888b7377f1591a6a165553730949f485977"}
Jul 10 20:47:19.337935 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.337882    2580 scope.go:117] "RemoveContainer" containerID="2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d"
Jul 10 20:47:19.346773 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.346740    2580 scope.go:117] "RemoveContainer" containerID="2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d"
Jul 10 20:47:19.347044 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:47:19.347026    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d\": container with ID starting with 2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d not found: ID does not exist" containerID="2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d"
Jul 10 20:47:19.347098 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.347054    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d"} err="failed to get container status \"2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d\": rpc error: code = NotFound desc = could not find container \"2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d\": container with ID starting with 2a9c4966eb43312c1242832e23e394b3b87154e3e063751c89220c047329709d not found: ID does not exist"
Jul 10 20:47:19.355961 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.355937    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"]
Jul 10 20:47:19.360000 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:19.359978    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-75e636-5ccff7b7bf-txbwc"]
Jul 10 20:47:21.281522 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:21.281488    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" path="/var/lib/kubelet/pods/b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588/volumes"
Jul 10 20:47:25.535925 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.535838    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"]
Jul 10 20:47:25.536382 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.536040    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" podUID="596240c9-9547-4f6b-b3b4-80e88dadff7c" containerName="maas-api" containerID="cri-o://e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097" gracePeriod=30
Jul 10 20:47:25.796763 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.796695    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:47:25.856008 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.855975    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hx8pq\" (UniqueName: \"kubernetes.io/projected/596240c9-9547-4f6b-b3b4-80e88dadff7c-kube-api-access-hx8pq\") pod \"596240c9-9547-4f6b-b3b4-80e88dadff7c\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") "
Jul 10 20:47:25.856008 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.856011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596240c9-9547-4f6b-b3b4-80e88dadff7c-openshift-service-ca\") pod \"596240c9-9547-4f6b-b3b4-80e88dadff7c\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") "
Jul 10 20:47:25.856230 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.856044    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls\") pod \"596240c9-9547-4f6b-b3b4-80e88dadff7c\" (UID: \"596240c9-9547-4f6b-b3b4-80e88dadff7c\") "
Jul 10 20:47:25.856408 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.856384    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/596240c9-9547-4f6b-b3b4-80e88dadff7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "596240c9-9547-4f6b-b3b4-80e88dadff7c" (UID: "596240c9-9547-4f6b-b3b4-80e88dadff7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 20:47:25.858161 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.858135    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596240c9-9547-4f6b-b3b4-80e88dadff7c-kube-api-access-hx8pq" (OuterVolumeSpecName: "kube-api-access-hx8pq") pod "596240c9-9547-4f6b-b3b4-80e88dadff7c" (UID: "596240c9-9547-4f6b-b3b4-80e88dadff7c"). InnerVolumeSpecName "kube-api-access-hx8pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 20:47:25.858266 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.858133    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "596240c9-9547-4f6b-b3b4-80e88dadff7c" (UID: "596240c9-9547-4f6b-b3b4-80e88dadff7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 20:47:25.956955 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.956910    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx8pq\" (UniqueName: \"kubernetes.io/projected/596240c9-9547-4f6b-b3b4-80e88dadff7c-kube-api-access-hx8pq\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:47:25.956955 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.956951    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/596240c9-9547-4f6b-b3b4-80e88dadff7c-openshift-service-ca\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:47:25.957173 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:25.956967    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/596240c9-9547-4f6b-b3b4-80e88dadff7c-maas-api-tls\") on node \"ip-10-0-133-106.ec2.internal\" DevicePath \"\""
Jul 10 20:47:26.377043 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.377008    2580 generic.go:356] "Generic (PLEG): container finished" podID="596240c9-9547-4f6b-b3b4-80e88dadff7c" containerID="e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097" exitCode=0
Jul 10 20:47:26.377233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.377055    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"
Jul 10 20:47:26.377233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.377083    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" event={"ID":"596240c9-9547-4f6b-b3b4-80e88dadff7c","Type":"ContainerDied","Data":"e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097"}
Jul 10 20:47:26.377233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.377131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn" event={"ID":"596240c9-9547-4f6b-b3b4-80e88dadff7c","Type":"ContainerDied","Data":"c934becb3e9374478287e950c8c59d80878e40b52594f446e63f74dd2593c3a8"}
Jul 10 20:47:26.377233 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.377153    2580 scope.go:117] "RemoveContainer" containerID="e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097"
Jul 10 20:47:26.386409 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.386391    2580 scope.go:117] "RemoveContainer" containerID="e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097"
Jul 10 20:47:26.386703 ip-10-0-133-106 kubenswrapper[2580]: E0710 20:47:26.386662    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097\": container with ID starting with e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097 not found: ID does not exist" containerID="e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097"
Jul 10 20:47:26.386769 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.386712    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097"} err="failed to get container status \"e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097\": rpc error: code = NotFound desc = could not find container \"e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097\": container with ID starting with e60c00f5b725f9f2f82edabfca8f1f2fe2cbf7d0d4d2196366b9b28e6203d097 not found: ID does not exist"
Jul 10 20:47:26.398211 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.398179    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"]
Jul 10 20:47:26.400386 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:26.400360    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-1b46b3-8ffd6c7d6-8bdxn"]
Jul 10 20:47:27.282169 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:27.282137    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596240c9-9547-4f6b-b3b4-80e88dadff7c" path="/var/lib/kubelet/pods/596240c9-9547-4f6b-b3b4-80e88dadff7c/volumes"
Jul 10 20:47:36.983419 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:36.983386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56c7cffc4c-lcfjd_1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4/manager/0.log"
Jul 10 20:47:37.563353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:37.563324    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5767ffdff4-qkqf6_173bfd55-e541-4e2c-871b-3beb2747e88e/copy-manifests/0.log"
Jul 10 20:47:37.577433 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:37.577401    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5767ffdff4-qkqf6_173bfd55-e541-4e2c-871b-3beb2747e88e/manager/0.log"
Jul 10 20:47:37.916303 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:37.916226    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-fdf5c9f6f-lshj4_772ffaa3-d62c-4fab-b85e-fe7fc0d03526/maas-api/0.log"
Jul 10 20:47:38.025873 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:38.025843    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xk7v9_e46b6254-9c5f-48ad-a66c-98344d3c990d/postgres/0.log"
Jul 10 20:47:38.786090 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:38.786062    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll_9b99e262-2fe2-46ac-ad69-7b8674fad8b5/extract/0.log"
Jul 10 20:47:38.791143 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:38.791121    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll_9b99e262-2fe2-46ac-ad69-7b8674fad8b5/util/0.log"
Jul 10 20:47:38.796975 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:38.796957    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll_9b99e262-2fe2-46ac-ad69-7b8674fad8b5/pull/0.log"
Jul 10 20:47:38.907897 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:38.907872    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls_355e96c3-2a65-4e69-9818-c73870010ea0/util/0.log"
Jul 10 20:47:38.913561 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:38.913537    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls_355e96c3-2a65-4e69-9818-c73870010ea0/pull/0.log"
Jul 10 20:47:38.919549 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:38.919530    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls_355e96c3-2a65-4e69-9818-c73870010ea0/extract/0.log"
Jul 10 20:47:39.036741 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:39.036637    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5_abf734a4-c170-4437-8a61-a23790b66e00/util/0.log"
Jul 10 20:47:39.042887 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:39.042867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5_abf734a4-c170-4437-8a61-a23790b66e00/pull/0.log"
Jul 10 20:47:39.048756 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:39.048739    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5_abf734a4-c170-4437-8a61-a23790b66e00/extract/0.log"
Jul 10 20:47:39.160004 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:39.159976    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc_78375b72-580e-4f47-a6f4-226c88ee9629/pull/0.log"
Jul 10 20:47:39.165991 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:39.165967    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc_78375b72-580e-4f47-a6f4-226c88ee9629/extract/0.log"
Jul 10 20:47:39.171859 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:39.171834    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc_78375b72-580e-4f47-a6f4-226c88ee9629/util/0.log"
Jul 10 20:47:40.563079 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:40.563043    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l7gcj_624e411b-2dbe-4a6e-8797-9d0c4ad44a35/discovery/0.log"
Jul 10 20:47:41.009133 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:41.009055    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-dwd4b_b66e1374-a44f-4dcd-a680-73a4c599fab7/payload-pre-processing/0.log"
Jul 10 20:47:41.120923 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:41.120895    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-4c54d_add741dc-4042-4fdc-8a77-2d024a3280d2/payload-processing/0.log"
Jul 10 20:47:41.556405 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:41.556368    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bksm8_3806b8a5-3878-4896-9df1-070f3d9b6f3d/main/0.log"
Jul 10 20:47:41.778146 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:41.778121    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xhq5t_9ffeb654-0465-476c-ad30-c0f954c16e98/main/0.log"
Jul 10 20:47:49.216245 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:49.216214    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-szwhd_eb9b3be5-7e24-4e37-9f4b-f7bf7d8cab64/global-pull-secret-syncer/0.log"
Jul 10 20:47:49.295353 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:49.295318    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g7kn9_44056a08-41c4-4faf-a8d3-d66637bef925/konnectivity-agent/0.log"
Jul 10 20:47:49.378731 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:49.378708    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-106.ec2.internal_b4dc4522d52b12e6a46164476b50912d/haproxy/0.log"
Jul 10 20:47:54.031142 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.031097    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll_9b99e262-2fe2-46ac-ad69-7b8674fad8b5/extract/0.log"
Jul 10 20:47:54.055709 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.055686    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll_9b99e262-2fe2-46ac-ad69-7b8674fad8b5/util/0.log"
Jul 10 20:47:54.079640 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.079597    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7592hsll_9b99e262-2fe2-46ac-ad69-7b8674fad8b5/pull/0.log"
Jul 10 20:47:54.106666 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.106639    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls_355e96c3-2a65-4e69-9818-c73870010ea0/extract/0.log"
Jul 10 20:47:54.128031 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.128003    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls_355e96c3-2a65-4e69-9818-c73870010ea0/util/0.log"
Jul 10 20:47:54.149218 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.149192    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e097hls_355e96c3-2a65-4e69-9818-c73870010ea0/pull/0.log"
Jul 10 20:47:54.185303 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.185272    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5_abf734a4-c170-4437-8a61-a23790b66e00/extract/0.log"
Jul 10 20:47:54.210297 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.210272    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5_abf734a4-c170-4437-8a61-a23790b66e00/util/0.log"
Jul 10 20:47:54.236719 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.236693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rsqd5_abf734a4-c170-4437-8a61-a23790b66e00/pull/0.log"
Jul 10 20:47:54.265685 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.265645    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc_78375b72-580e-4f47-a6f4-226c88ee9629/extract/0.log"
Jul 10 20:47:54.288242 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.288172    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc_78375b72-580e-4f47-a6f4-226c88ee9629/util/0.log"
Jul 10 20:47:54.316577 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:54.316551    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xdzxc_78375b72-580e-4f47-a6f4-226c88ee9629/pull/0.log"
Jul 10 20:47:57.112723 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.112632    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a251b7a8-84c4-4d1c-a731-afc7568c32e7/alertmanager/0.log"
Jul 10 20:47:57.130645 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.130622    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a251b7a8-84c4-4d1c-a731-afc7568c32e7/config-reloader/0.log"
Jul 10 20:47:57.153210 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.153185    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a251b7a8-84c4-4d1c-a731-afc7568c32e7/kube-rbac-proxy-web/0.log"
Jul 10 20:47:57.180645 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.180612    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a251b7a8-84c4-4d1c-a731-afc7568c32e7/kube-rbac-proxy/0.log"
Jul 10 20:47:57.201010 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.200981    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a251b7a8-84c4-4d1c-a731-afc7568c32e7/kube-rbac-proxy-metric/0.log"
Jul 10 20:47:57.224133 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.224106    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a251b7a8-84c4-4d1c-a731-afc7568c32e7/prom-label-proxy/0.log"
Jul 10 20:47:57.245119 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.245088    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a251b7a8-84c4-4d1c-a731-afc7568c32e7/init-config-reloader/0.log"
Jul 10 20:47:57.319081 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.319048    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-t2c72_2c4bc253-6422-4be4-83d4-6bb829538a24/kube-state-metrics/0.log"
Jul 10 20:47:57.340372 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.340347    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-t2c72_2c4bc253-6422-4be4-83d4-6bb829538a24/kube-rbac-proxy-main/0.log"
Jul 10 20:47:57.360700 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.360658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-t2c72_2c4bc253-6422-4be4-83d4-6bb829538a24/kube-rbac-proxy-self/0.log"
Jul 10 20:47:57.442304 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.442234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5v6j2_e68fbbfc-8e68-49fa-8725-9efb03195902/node-exporter/0.log"
Jul 10 20:47:57.462133 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.462107    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5v6j2_e68fbbfc-8e68-49fa-8725-9efb03195902/kube-rbac-proxy/0.log"
Jul 10 20:47:57.483281 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.483258    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5v6j2_e68fbbfc-8e68-49fa-8725-9efb03195902/init-textfile/0.log"
Jul 10 20:47:57.936136 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.936106    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-w8gwm_c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220/prometheus-operator/0.log"
Jul 10 20:47:57.955563 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:57.955520    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-w8gwm_c3d6a5e6-2c93-4b16-a6fb-80b68fe8d220/kube-rbac-proxy/0.log"
Jul 10 20:47:58.017638 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:58.017616    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b6ddc97b-2ttrw_9c35ba24-bb48-420b-9c04-57c89b1f6a80/telemeter-client/0.log"
Jul 10 20:47:58.041620 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:58.041596    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b6ddc97b-2ttrw_9c35ba24-bb48-420b-9c04-57c89b1f6a80/reload/0.log"
Jul 10 20:47:58.064227 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:58.064194    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85b6ddc97b-2ttrw_9c35ba24-bb48-420b-9c04-57c89b1f6a80/kube-rbac-proxy/0.log"
Jul 10 20:47:59.424620 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.424582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"]
Jul 10 20:47:59.425018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.424965    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" containerName="maas-api"
Jul 10 20:47:59.425018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.424978    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" containerName="maas-api"
Jul 10 20:47:59.425018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.424988    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596240c9-9547-4f6b-b3b4-80e88dadff7c" containerName="maas-api"
Jul 10 20:47:59.425018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.424993    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="596240c9-9547-4f6b-b3b4-80e88dadff7c" containerName="maas-api"
Jul 10 20:47:59.425018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.425000    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" containerName="main"
Jul 10 20:47:59.425018 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.425005    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" containerName="main"
Jul 10 20:47:59.425216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.425058    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="596240c9-9547-4f6b-b3b4-80e88dadff7c" containerName="maas-api"
Jul 10 20:47:59.425216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.425066    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa73ed9c-3d4c-4fef-ad15-59a7dc7baa5e" containerName="main"
Jul 10 20:47:59.425216 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.425083    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2efbdbc-ca51-48d8-84d7-a2d2d8b1d588" containerName="maas-api"
Jul 10 20:47:59.428350 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.428335    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.431037 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.431014    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xk8xm\"/\"default-dockercfg-jnk86\""
Jul 10 20:47:59.431178 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.431068    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xk8xm\"/\"openshift-service-ca.crt\""
Jul 10 20:47:59.431178 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.431070    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xk8xm\"/\"kube-root-ca.crt\""
Jul 10 20:47:59.436120 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.436096    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"]
Jul 10 20:47:59.461804 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.461775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-podres\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.461970 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.461840    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-lib-modules\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.461970 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.461892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-sys\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.461970 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.461919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhp72\" (UniqueName: \"kubernetes.io/projected/ca227360-1f0e-47d1-9888-597085cc4204-kube-api-access-hhp72\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.461970 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.461940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-proc\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563059 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-lib-modules\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-sys\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhp72\" (UniqueName: \"kubernetes.io/projected/ca227360-1f0e-47d1-9888-597085cc4204-kube-api-access-hhp72\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-proc\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-podres\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-sys\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-proc\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-lib-modules\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.563275 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.563278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca227360-1f0e-47d1-9888-597085cc4204-podres\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.570652 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.570624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhp72\" (UniqueName: \"kubernetes.io/projected/ca227360-1f0e-47d1-9888-597085cc4204-kube-api-access-hhp72\") pod \"perf-node-gather-daemonset-6rpfl\" (UID: \"ca227360-1f0e-47d1-9888-597085cc4204\") " pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.740246 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.740159    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:47:59.874714 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:47:59.874662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"]
Jul 10 20:47:59.876735 ip-10-0-133-106 kubenswrapper[2580]: W0710 20:47:59.876708    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podca227360_1f0e_47d1_9888_597085cc4204.slice/crio-d6aa2e384c4ce36ca1e05a46165b955487aafeaa40068338f96df1b6fb4fe90e WatchSource:0}: Error finding container d6aa2e384c4ce36ca1e05a46165b955487aafeaa40068338f96df1b6fb4fe90e: Status 404 returned error can't find the container with id d6aa2e384c4ce36ca1e05a46165b955487aafeaa40068338f96df1b6fb4fe90e
Jul 10 20:48:00.551331 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:00.551285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl" event={"ID":"ca227360-1f0e-47d1-9888-597085cc4204","Type":"ContainerStarted","Data":"ea798f2cdd0dfd1b220cb982270d6d3e7f90a3cc90dfe4b408274ac00fd5d25a"}
Jul 10 20:48:00.551752 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:00.551354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl" event={"ID":"ca227360-1f0e-47d1-9888-597085cc4204","Type":"ContainerStarted","Data":"d6aa2e384c4ce36ca1e05a46165b955487aafeaa40068338f96df1b6fb4fe90e"}
Jul 10 20:48:00.551752 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:00.551463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:48:00.567827 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:00.567782    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl" podStartSLOduration=1.567766585 podStartE2EDuration="1.567766585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 20:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 20:48:00.56721942 +0000 UTC m=+2367.792986350" watchObservedRunningTime="2026-07-10 20:48:00.567766585 +0000 UTC m=+2367.793533513"
Jul 10 20:48:02.139394 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:02.139362    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89fvm_2d0236af-836d-44dc-ab84-dbfba301a7f8/dns/0.log"
Jul 10 20:48:02.159551 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:02.159524    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89fvm_2d0236af-836d-44dc-ab84-dbfba301a7f8/kube-rbac-proxy/0.log"
Jul 10 20:48:02.274797 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:02.274771    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cbbbn_edfc10a5-c760-4fb3-8ed1-49e0bffd7f80/dns-node-resolver/0.log"
Jul 10 20:48:02.856202 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:02.856176    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7898cfcbf4-62vs7_76fad17c-35dc-4aa6-b566-cfb59a318a33/registry/0.log"
Jul 10 20:48:02.927017 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:02.926987    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zzsd6_d89da9d9-5f26-4757-90ee-cf3daed84967/node-ca/0.log"
Jul 10 20:48:03.987432 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:03.987400    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l7gcj_624e411b-2dbe-4a6e-8797-9d0c4ad44a35/discovery/0.log"
Jul 10 20:48:04.151839 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:04.151799    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-dwd4b_b66e1374-a44f-4dcd-a680-73a4c599fab7/payload-pre-processing/0.log"
Jul 10 20:48:04.185708 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:04.185629    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-4c54d_add741dc-4042-4fdc-8a77-2d024a3280d2/payload-processing/0.log"
Jul 10 20:48:04.912399 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:04.912371    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fl6wl_39306aff-c570-417f-9d22-7db335d5909e/serve-healthcheck-canary/0.log"
Jul 10 20:48:05.580397 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:05.580367    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vds78_a845fc09-5764-45bb-9a1f-2270ed43222a/kube-rbac-proxy/0.log"
Jul 10 20:48:05.602588 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:05.602561    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vds78_a845fc09-5764-45bb-9a1f-2270ed43222a/exporter/0.log"
Jul 10 20:48:05.623117 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:05.623089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vds78_a845fc09-5764-45bb-9a1f-2270ed43222a/extractor/0.log"
Jul 10 20:48:06.566280 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:06.566254    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xk8xm/perf-node-gather-daemonset-6rpfl"
Jul 10 20:48:08.156710 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:08.156657    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56c7cffc4c-lcfjd_1d2bcd25-2978-4d64-bd73-a7e4b6f72ae4/manager/0.log"
Jul 10 20:48:08.472503 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:08.472470    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5767ffdff4-qkqf6_173bfd55-e541-4e2c-871b-3beb2747e88e/manager/0.log"
Jul 10 20:48:08.492205 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:08.492179    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5767ffdff4-qkqf6_173bfd55-e541-4e2c-871b-3beb2747e88e/copy-manifests/0.log"
Jul 10 20:48:09.909274 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:09.909174    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bd7b6bf4b-xf2jm_b90491f6-a35b-4bba-8eea-c1eaefcb8c38/manager/0.log"
Jul 10 20:48:10.391435 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:10.391385    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l7gcj_624e411b-2dbe-4a6e-8797-9d0c4ad44a35/discovery/0.log"
Jul 10 20:48:10.501995 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:10.501967    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-dwd4b_b66e1374-a44f-4dcd-a680-73a4c599fab7/payload-pre-processing/0.log"
Jul 10 20:48:10.522658 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:10.522621    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-4c54d_add741dc-4042-4fdc-8a77-2d024a3280d2/payload-processing/0.log"
Jul 10 20:48:11.898714 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:11.898662    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-l7gcj_624e411b-2dbe-4a6e-8797-9d0c4ad44a35/discovery/0.log"
Jul 10 20:48:12.016835 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:12.016803    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-dwd4b_b66e1374-a44f-4dcd-a680-73a4c599fab7/payload-pre-processing/0.log"
Jul 10 20:48:12.037980 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:12.037933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-4c54d_add741dc-4042-4fdc-8a77-2d024a3280d2/payload-processing/0.log"
Jul 10 20:48:16.637739 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.637658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzdm9_8620a41e-4cb4-4a0b-b54d-ef177115f6b3/kube-multus-additional-cni-plugins/0.log"
Jul 10 20:48:16.659479 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.659450    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzdm9_8620a41e-4cb4-4a0b-b54d-ef177115f6b3/egress-router-binary-copy/0.log"
Jul 10 20:48:16.679874 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.679848    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzdm9_8620a41e-4cb4-4a0b-b54d-ef177115f6b3/cni-plugins/0.log"
Jul 10 20:48:16.701301 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.701274    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzdm9_8620a41e-4cb4-4a0b-b54d-ef177115f6b3/bond-cni-plugin/0.log"
Jul 10 20:48:16.722857 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.722836    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzdm9_8620a41e-4cb4-4a0b-b54d-ef177115f6b3/routeoverride-cni/0.log"
Jul 10 20:48:16.743505 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.743481    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzdm9_8620a41e-4cb4-4a0b-b54d-ef177115f6b3/whereabouts-cni-bincopy/0.log"
Jul 10 20:48:16.763510 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.763487    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzdm9_8620a41e-4cb4-4a0b-b54d-ef177115f6b3/whereabouts-cni/0.log"
Jul 10 20:48:16.942595 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:16.942487    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hpfkh_222d6857-2959-42ad-b364-7322ba3bcb8e/kube-multus/0.log"
Jul 10 20:48:17.034023 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:17.033995    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4wx2c_d9f1c3d3-65c9-4365-8dc8-0517ceae9708/network-metrics-daemon/0.log"
Jul 10 20:48:17.057397 ip-10-0-133-106 kubenswrapper[2580]: I0710 20:48:17.057370    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4wx2c_d9f1c3d3-65c9-4365-8dc8-0517ceae9708/kube-rbac-proxy/0.log"